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ABSTRACT  (UNCLASSIFIED) 


This  study  containM"^ summary  of  the  results  of  the  technology  project  QUEST:  Quality  of 
Expert  Systems  ,  carried  out  under  commission  of  the  Dutch  Ministry  of  Defence,  Directorate 
Defense  Research  and  Development.  Participants  in  the  project  are  TNO  Physics  and  Electronics 
Laboratory  (TNO-FEL),  University  of  Limburg  (RL)  and  the  Research  Institute  for  Knowledge 
Systems  (RIKS).  After  an  analysis  of  the  problems  encountered  in  expert  systems  development,  a 
quality  framework  is  developed  which  takes  a  view  at  the  quality  problem  from  three 
()erspectives;  the  quality  of  the  development  process,  the  quality  of  the  specifications  and  the 
quality  of  the  expert  system  viewed  as  a  product.  In  order  to  get  a  better  grasp  of  the  problem  a 
number  of  methods  and  techniques,  derived  from  conventional  and  artificial  intelligence  systems 
development,  are  reviewed.  Secondly,  the  conceptual  similarities  between  databases  and 
knowledgebases  are  stressed.  The  use  of  conventional  specification  methods,  in  particular 
Nijssens  Information  Analysis  Methodology  (NIAM),  is  considered.  In  addition  to  this  algorithms 
for  preserving  consistency  and  integrity  of  the  knowledgebase  are  compared.  Thirdly  the 
modularity  and  structure  of  knowledgebases  is  examined,  together  with  the  ai^licability  of 
conventional  testing  methodologies  in  expert  systems.  Lastly  it  is  demonstrated  that  the 
integration  of  database  theory  and  artificial  intelligence  signifies  a  stq>  in  the  direction  of  a  better 
quality  control  of  expert  systems.  „ 
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SAMENVATTING  (ONGERUBRICEERD) 

Dit  rapport  bevat  een  samenvatting  van  de  resultaten  van  het  technologieproject  "QUEST; 
Kwaliteit  van  Expertsystemen",  uitgevoerd  in  opdracht  van  het  Nederlandse  Ministerie  van 
Defensie,  Dircctoraat-Generaal  Wetenschappelijk  Onderzoek  en  Ontwikkeling.  Deelnemers  aan 
het  project  zijn  het  Fysisch  en  Elektronisch  Laboratorium  TNO  (FEL-TNO),  de  Rijksuniversiteit 
Limburg  (RL)  en  het  Research  Insdtuut  voor  Kennis-Systemen  (RIKS).  Na  een  analyse  van  de 
problemen  die  zich  voordoen  bij  de  ontwikkeling  van  expertsystemen  woidt  een  raamweik 
ontwikkeld  dat  het  kwaliteits-probleem  benaderd  uit  drie  gezichtspunten:  de  kwaliteit  van  het 
ontwikkelproces,  de  kwaliteit  van  de  specihcaties  en  de  kwaliteit  van  het  expertsysteem  als 
produkt.  Om  een  beter  begrip  van  het  probleem  tc  krijgen  worden  methoden  en  technieken 
besproken  om  de  kwaliteit  van  systemen  te  beheersen.  Daaibij  wordt  zowel  verwezen  naar 
conventionele  als  kunstmatig  intelligente  systeemontwikkeling.  Ten  tweede  is  er  aandacht  voor  de 
conceptuele  overeenkomsten  tussen  gegevensbanken  en  kennisbanken.  Implicatie  hiervan  is  het 
gebruik  van  Nijssens  Informatie-Analyse  Methode  (NIAM)  als  specificatiemethode  voor 
kennisbanken.  Verschillende  algoritmen  gericht  op  het  waarborgen  van  consistentie  en 
volledigheid  van  de  knowledgebase  worden  met  elkaar  vergeleken.  Ten  derde  is  het  onderwerp 
modulariteit  en  suticturering  van  kennisbanken  onderzocht  met  daamaast  aandacht  voor  de 
toepasbaartieid  van  conventionele  testmethodologieCn  op  expertsysteem-software.  Ten  slotte 
wordt  aangetoond  dat  met  de  integratie  van  gegevensbanktheorie  en  kuitstmatige  intelligentie  een 
belangrijke  stap  kan  worden  gezet  op  het  pad  naar  een  betere  kwaliteitsbeheersing  van 
kennissystemen. 
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1  INTRODUCTION 

Since  the  fifties,  research  in  the  field  of  artificial  intelligence  (AI)  aims  at  the  development  of 
systems  that  manifest  themselves  in  new  functions  such  as  visual  perception,  natural  language 
processing,  learning  and  expert  systems.  The  objective  is  to  capture  the  input-ou^mt  behaviour  of 
the  system  with  regard  to  these  specific  tasks,  that  can  match  its  performance  with  that  of  a 
human  being.  This  comes  down  to  modeUing  cognitive  processes  in  such  a  way  that  these  can  be 
performed  by  a  computer  system.  This  activity  resembles  the  modelling  of  other  (technical) 
systems,  that  are  familiar  to  science.  The  specifications  of  a  system,  for  instance,  determine  the 
level  of  detail  and  (un)cettainty  required  by  the  model,  both  for  static  and  dynamic  behaviour. 

However,  there  is  an  essential  difference  between  the  accepted  concepts,  methods  and  tools.  The 
construction  of  qualitative  models  lies  at  die  centre,  i.e.  models  that  establish  explicit 
specifications  of  the  (problem)  domain  and  the  solutions/heuristics  in  terms  of  facts  and  their 
relevant  interrelations,  rather  than  numerical  descriptions.  To  execute  these  models,  the 
information  processing  ciqiabilities  of  the  computer  are  extended  with  a  derivation  component, 
usually  based  on  logic. 

It  is  expeaed  that  conventional,  automated  systems  will  in  future  contain  AI  components. 
Reliability  amongst  other  things,  must  be  assured  if  these  systems  are  to  be  used  for  critical  tasks. 
It  is  a  well-known  fact  that  the  development  of  large  systems  is  plagued  by  exceeding  the  project 
funds  and  delivery  dates.  Then,  when  the  system  obtains  operational  status,  it  often  turns  out  to  be 
fuiKtionally  defective.  Quality  research  receives  much  attention  in  the  fields  of  both  conventional 
and  AI  systems.  Very  stria  demands  are  put  on  military  and  critical  civil  applications,  especially 
those  concerning  reliatHlity  and  fail-safe  operations.  Examples  of  military  systems  ate,  for 
instance.  Command,  Control,  Communication  and  Intelligence  systems  (C^I);  critical  civil 
applications  ate  process  control  systems  for  chemical  and  power  idants. 

During  research  into  the  quality  of  autCHnated  systems,  one  is  omfixMited  with  a  number  of 
proMems.  Rtstly,  there  is  no  generally  accepted  definition  of  the  notion  of  quality.  The  factors 
that  make  up  the  quality  of  a  system  ate  impoitut  entities  to  determine  with  which  quality  can  be 
measured.  Secondly,  a  system’s  quality  is  measured  after  it  has  become  (^rational.  Defects 
appatem  in  this  phase,  can  only  be  staved  with  luge  investments,  both  in  time  and  money.  To 
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ensure  a  proper  quality  assurance  of  automated  systems,  it  is  necessary  to  take  this  aspect  into 
account  during  the  very  first  phases  of  system  development.  The  emergence  of  expert  systems  as 
a  commercial  product  stems  mainly  from  the  principle  of  software  reuse,  e.g.  in  expert  system 
shells.  Other  types  of  software  too,  such  as  relational  database  management  systems  (RDBMSs) 
have  benefited  from  this  principle.  In  this  study,  therefore,  expert  system  development  will 
primarily  be  viewed  as  knowledgebase  development 

This  study  attempts  to  present  a  survey  of  methods  and  techniques  availatde  to  control  the  quality 
of  expert  systems.  Three  aspects  related  to  the  quality-problem  can  be  distinguished,  i.e.:  the 
development  process,  the  specification  and  the  product.  This  study  focuses  at  that  part  of  system 
development  that  sets  expert  systems  substantially  apart  from  other  software:  the  specification 
and  implementation  of  the  knowledgebase.  The  methods  and  techniques  selected  for  the 
development  of  expert  systems  are  based  on,  or  are  directly  derived  from  (relational)  database 
technology.  Though  it  is  has  not  yet  been  established  that  the  relational  database  approach  is 
adequate  for  all  types  of  expert  systems,  it  seems  that  this  approach  offers  a  better  perspective 
where  quality  assurance  is  concerned,  than  any  other  development  methodology.  This  study 
delineates  a  quality  framework  in  terms  of  process,  specification  and  product,  on  the 
understanding  that  'the  process'  refers  to  the  comprehensive  process  of  expert  system 
development  (the  life  cycle),  whereas  'the  product'  and  'the  specification'  relate  to  parts  thereof, 
i.e.  the  knowledgebase  and  the  knowledge  model. 

Before  entering  the  subjects  of  the  tKxt  chapters,  it  is  important  to  know  what  exactly  makes  up 
an  expert  system.  Expert  systems  form  a  new  class  of  automated  systems  that  have  enjoyed  a 
large  interest  over  the  past  few  years.  They  form  one  of  the  practical  results  of  the  research  into 
the  field  of  artificial  intelligence. 

While  building  expert  systems,  methods  and  techniques  are  allied  to  create  man-machine 
systems  that  can  solve  problems  in  a  specific  area.  To  do  this,  knowledge,  experience  (or 
expertise  for  that  matter)  of  a  human  expert  is  needed.  Beside  theoretical  krwwledge  from 
textbooks,  this  may  also  exist  of  more  general  experience  rules  or  rules  of  thumb.  Another  name 
for  this  quite  vague  knowledge  is  heuristic.  Tleutistics  enable  the  human  expert  to  make,  where 
necessary,  educated  guesses,  to  recognise  promising  approaches  to  problems  and  work  effectively 
with  incomfdete  or  even  wrong  data’  (Hayes-Roth83]. 
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Knowledge  lies  at  the  centre  of  expert  systems.  A  number  of  reasons  can  be  given  for  this: 

•  certain  problems  can  haidly  be  solved  or  cannot  be  solved  at  all  with  the  help  of  fixed 
algorithms; 

•  a  person  qualifies  as  an  expert  because  of  the  very  fact  that  he  possesses  more  than  only 
theoretical  textbook  knowledge,  his  heuristics  so  to  say; 

•  knowledge  is  a  scarce  commodity  that  can  be  communicated  to  a  larger  group  of  persons 
by  means  of  an  expert  system. 

Expert  systems  are  based  on  symbolic  processing,  rather  than  conventional  systems  that 

emphasize  numerical  processing.  Pertinent  differences  are  shown  in  figure  1.1. 


Processing  method 

Symbolic 

Numerical 

Technique  used 

heuristic  search 

algorithmic 

Definition  of  solution  steps 

not  explicit 

explicit 

Answers 

satisfactory 

optimal 

Procedures  /  data 

separated 

interwoven 

Knowledge 

not  exact 

exact 

Adaption 

often 

seldom 

Figure  1.1:  DifferetKes  between  symbolic  and  numerical  processing. 

Expert  systems  usually  consist  of  four  elonents;  the  knowledgebase,  a  reasoning  mechanism,  an 
explanation  facility  and  a  man-machine  interface.  The  knowledgebase  holds  facts  and  rules  thiU 
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represent  the  actual  knowledge  of  the  system.  The  reasoning  mechanism  reflects  the  way  of 
reasoning  used  in  a  specific  proUem  area.  It  consists  of  procedures  with  which  new  knowledge 
can  be  derived  on  the  basis  of  available  knowledge  (or  information).  Expert  systems  offer  the  user 
advice  with  regard  to  solving  certain  problems.  The  user’s  confidence  in  this  advice  is 
strengthened  if  the  system  is  able  to  explain  how  the  actual  result  has  come  into  existence. 
Furthermore,  the  system  must  sometimes  query  the  user,  especially  when  it  is  not  able  to  elicit 
certain  data  from  the  knowledge  available.  At  these  moments,  a  user  wants  to  know  why  the 
question  arises.  From  the  foregoing,  it  tq>pears  that  the  user  interface  is  vital  to  the  system's 
functioning.  The  man-machine  interface  will  have  to  take  care  of  this. 

Expert  systems  are  used  for  various  problem  categories.  The  most  apparent  is  fault  diagnosis.  The 
word  fault  must  not  be  taken  too  literally;  especially  in  the  field  of  medical  diagnosis,  expert 
systems  have  come  into  their  own  successfully.  The  MYQN  system  [Shoitliffe76],  applied  for 
the  diagnosis  in  the  field  of  internal  medicine  can  be  seen  as  a  milestone  in  the  development. 
Other  applications  are  the  interpretation  of  data  to  describe  a  certain  situation  realistically,  as  for 
example,  signal  interpretation  systems  (sonar  and  radar).  Monitoring  systems  are  of  a  somewhat 
mote  difficult  type.  These  systems  are  meant  to  keep  an  eye  on  a  process.  This  involves 
measuring  process  parameters;  ^uld  the  values  measured  deviate  from  a  predefined  norm,  the 
system  will  diagnose  the  problem  after  which  possible  counteractions  will  be  planned.  However, 
planning  of  activities  is  a  difficult  task  for  an  artificial  intelligence  system.  Applications  of  this 
type  are  therefore  scarce. 

It  remains  hazardous  to  sharply  define  expert  systems.  It  may  be  compared  with  shooting  at  a 
moving  target.  The  moment  a  system  can  execute  a  certain  task,  the  intelligence  needed  for 
execution  is  sometimes  forgotten. 

Especially  [Nijssen86]  expands  on  the  thesis  that  expert  systems  are  nothing  but  automated 
information  systems  cotttaining  a  certain  amount  of  human  expertise.  This  expertise  (or 
knowledge)  is  merely  a  set  of  interrelated  facts.  Facts  that  could  be  entered  by  the  user  himself  (as 
the  result  of  a  query  or  during  the  creation  of  the  database).  It  is  also  possible  that  diey  were 
derived  from  facts  already  present  in  the  database  with  the  help  of  derivation  rules.  Lastly,  they 
can  be  the  result  of  a  reasoning  process:  by  applying  inference  rules  to  the  facts  already  present, 
new  facts  can  be  inferred.  Figure  1.2  shows  the  structure  of  what  can  be  termed  a  relational  expert 
system.  It  demonstrates  the  equivaletKe  of  databases  and  knowledgebases. 
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A  relational  expert  system  has  a  structural  section  containing  the  definitions  of  the  tables,  a 
section  with  validation  niles,  a  section  with  derivation  rules  (that  can  infer  facts  that  are  implicitly 
contained  in  the  database)  and  a  section  with  inference  rules  (that  can  infer  facts  that  are  i»t 
implicitly  contained  in  the  database).  Users  converse  with  the  system  and  operate  their  own 
database,  next  to  that  there  is  a  virtual  database  containing  derived  and  inferred  facts. 


Figure  1 .2;  A  relational  expert  system. 


Conceits  such  as  data  independence,  data  integrity,  controUed  redundatKy,  security  and  privacy 
that  are  used  in  an  RDBMS  are  also  important  to  expert  systems.  Other  reasons  to  hold  cm  to  the 
relational  model  are  its  conceptual  simplicity  and  the  fact  that  system  developers  are  familiar  with 
it.  By  using  an  RDBMS,  update  anomalies  can  be  prevorted,  just  like  redundant  storage. 
Moreover,  it  offers  a  flexible  growth  potential  when  operational  concepts  change  and  alterations 
must  be  made  to  the  data  structure. 
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By  considering  a  knowledgebase  as  a  special  kind  of  database,  various  facilities  available  in 
DBMSs,  can  also  be  used  in  knowledgebase  management  systems  (KBMSs).  Examples  are 
recovery  (the  repair  of  a  database  after  a  calamity,  an  error  or  power  failure),  concurrence  (the 
simultaneous  use  of  a  database  by  aifterent  persons),  security  (the  protection  of  a  database  against 
unauthorized  use)  and  integrity  (guarding  the  consistency  of  the  database).  Especially  with  regard 
to  the  latter  point,  a  direct  relationship  can  be  made  with  analysis  and  design  methods  for 
databases.  Globally,  this  implies  that  a  conceptual  model  of  the  knowledge  domain  must  be  made. 
The  conceptual  model  lies  somewhere  in  between  the  internal  model  of  the  database  (the  way  in 
which  relationships  are  physically  incorporated  into  the  database)  and  the  external  model  (the  way 
in  which  the  user  views  the  system).  The  conceptual  model  must  be  a  complete  and  consistent 
representation  of  the  knowledge  domain,  where  a  distinction  is  made,  just  as  in  a  database 
application,  between  a  knowledge  scheme  (definitions  of  all  facts  and  relations  present)  and  the 
actual  knowledgebase.  This  distinction  can  also  be  taken  as  a  clear  separation  between  types  and 
instances.  The  advantage  is  that  the  knowledgebase  can  now  be  discussed  on  an  abstract  level. 
Consistency  and  completeness  can  be  monitored  by  using  constraints  that  are  applied  to  the  actual 
knowledgebase. 

The  reports  on  which  this  outline  is  based  are  [PEL  1989- 148],  [FEL-89-A267]  and  [FEL-90- 
A312]. 

The  remaining  chapters  of  this  study  deal  with  the  following  subjects.  (Chapter  2  contains  a  further 
analysis  of  the  quality  problem  in  expert  systems.  On  the  basis  of  this  a  quality  framework  is 
developed,  departing  from  three  approaches:  i.c.  process,  specification  and  product  with  their 
related  quality  criteria.  Chapters  3, 4  and  5  elaborate  on  these  approaches.  Chapter  6  deals  with  an 
aspect  of  expert  systems  that  is  essential  to  quality  assurance,  i.e.  the  integrity  of  the 
knowledgebase.  Finally,  chi^r  7  contains  the  coiKlusions. 
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2  A  QUALITY  FRAMEWORK 

2. 1  Introduction 

The  quality  of  expert  systems  is  generally  regarded  as  disappointing,  certainly  when  compared  to 
more  conventional  software.  In  particular  knowledge  acquisition,  testing,  evaluation  and 
maintenance  of  knowledgebases  are  deemed  to  be  problematical  aspects.  As  yet,  there  is  little 
agreement  when  it  comes  to  the  way  in  which  these  problems  should  be  dealt  with.  An  important 
motivation  for  research  into  the  quality  of  expert  systems  is  the  fact  that  conventional  systems 
make  increasing  use  of  intelligent  modules,  without  complying  to  the  same,  strict  quality 
requirements  that  are  put  on  the  conventional  part. 

A  number  of  aspects  influence  the  requirements  related  to  quality.  Type,  application  area  and 
scope  of  the  system  come  to  mind.  If  the  system  is  meant  to  serve  as  a  research  prototype,  then  it 
will  neet',  less  stringent  quality  requirements  than  an  operational  system  does.  The  role  of  the 
application  area  is  also  important,  especially  in  the  case  of  critical  applications,  such  as  process 
control  systems  in  chemical  industry  or  power  plants.  The  same  goes  for  military  command 
systems,  usually  termed  C^I  systems  (Command,  Control,  Communications  and  Intelligence)  in 
military  jargon  [Napheys89).  Such  a  system  needs  more  stringent  quality  requirements  than  an 
administrative  one.  Automated  systems  sometimes  are  very  large  and  therefore  difficult  to 
oversee.  Information  analysts  must  communicate  system  specifications  to  programmers.  Using  a 
quality  standard  assists  in  controlling  such  a  project. 

This  chapter  first  presents  an  analysis  of  the  problems  that  arc  encountered  in  controlling  the 
quality  of  expert  systems.  Next,  a  quality  framework  will  be  developed,  with  a  number  of  quality 
criteria  related  to  it 


2.2  Problem  analysts 

(^ality  assurance  performed  along  the  lines  of  the  framework  presented  for  this  purpose  can 
contribute  to  the  development  of  better  systems,  (^ality  assurance  implies  the  realization  of  a 
good  product  (eventually  to  be  measured  through  user  appreciation)  as  well  as  meeting  the  ts 
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established  time-related  and  financial  ctmditions.  The  quality  assurance  of  expert  systems  leaves 
much  to  be  desired,  as  is  apparent  from  a  multitude  of  complaints,  such  as: 

•  frequently  exceeding  the  budget; 

•  disappointing  final  produa,  e.g.  in  the  area  of: 

•  connectivity  with  other  (conventional)  software; 

•  flexible  user  interaction; 

•  lack  of  maintainability  of  the  product. 

Such  a  list  of  complaints  is  not  substantial  enough  to  evaluate  the  seriousness  of  the  problems,  let 
alone  to  determine  countermeasures.  To  investigate  in  how  far  these  complaints  are  specific  or 
generic  for  expert  systems,  it  is  necessary  to  relate  them  to  the  problems  that  lie  at  the  root  of 
them.  Making  an  inventory  of  the  activities  of  the  development  process  where  most  problems 
occur,  is  the  first  step.  After  that,  the  next  can  be  identified: 

•  specification; 

•  knowledge  acquisition  (=knowledge  acquisition  +  modelling); 

•  evaluation; 

•  testing; 

•  maintenance. 

The  all-embracing  problem  in  the  development  of  many  expert  systems  is  the  absence  of  clear, 
hierarchically  refined  specifications.  This  is  especially  true  for  so-called  expert  emulation 
systems.  The  initial  issue,  the  emulation  of  some  kind  of  expert,  does  rx)t  lend  itself  very  well  to 
systematical  refinement.  This  is  probably  partly  due  to  the  fact  that  methods  and  techniques  of  the 
AI  discipline  are  not  yet  fully  developed.  It  is  doubtful  whether  conventional  techniques  can  offer 
a  satisfactory  answer.  Automating  (or  effectively  supporting)  a  medical  diagnosis  is  more  difficult 
than  automating  a  corporation's  administration  because  of  the  considerably  lower  level  of  domain 
formalization  involved. 

Knowledge  acquisition  is  generally  considered  to  be  the  bottle-neck  of  expert  system 
development  [Breuker88a].  Many  expert  systems  developed  in  the  past  have  made  it  clear  that 
eliciting  knowledge  from  a  human  expert  and  making  it  a  coherent  entity,  is  a  laborious  and  time- 
consuming  process.  The  main  reasons  are  the  fact  that: 

•  knowledge  of  the  human  expert  is  mostly  compiled  (difficult  to  make  explicit); 

•  there  is  no  absolute  consensus  among  human  experts. 
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Evaluation  deals  with  the  functionality  of  the  system.  In  many  expert  systems,  the  evaluation  of 
the  accuracy  in  particular,  is  a  problem.  This  evaluation  problem  occurs  when  concrete  validation 
of  the  system's  output  is  not  possible,  because  the  cornea  answer  is  unknown  or  because  there  are 
more  (reasonably)  correct  answers.  Beside  the  direct  consequences  of  this  tiresome  evaluation  for 
quality  assurance,  there  also  is  an  indirect  effea.  Namely,  the  extra  requirements  with  regard  to 
the  functionality  of  tlie  system,  for  instance,  in  the  form  of  flexible  user  interaction  and  an 
explanation  that  is  readily  understood  by  the  user. 

Testing  capacity  is  related  to  the  evaluation  capacity.  However,  evaluation  is  a  matter  (or  at  least, 
could  be)  of  judging  the  final  product,  whereas  the  system’s  testing  capacity  is  inseparably 
connected  to  a  hierarchic  specification  with  a  corresponding,  systematic  subdivision  of  software 
into  smaller,  distinct  subsystems  [Myers78].  In  most  of  the  expert  systems  developed  so  far,  such 
a  hierarchical  structure  is  hardly  to  be  found.  Observations  like  'Expert  systems  often  have  a 
convoluted,  intricate  flow  of  control'  [LIinas87]  also  seem  to  point  in  this  direction.  Beside  the 
lack  of  hierarchical  structure,  the  use  of  heuristics  and  incalculable  entities  too,  frustrate  the 
testing  of  expert  systems. 

The  problem  of  maintenance  also  leads  back  to  what  principally  is  the  lack  of  hierarchy  as  well  as 
modularity  in  the  average  knowledgebase.  This  lack  is  especially  evident  for  those  domains  where 
knowledge  has  become  the  subject  of  frequent  change. 

Together  with  the  issue  of  maintenance,  the  defective  explanation  facilities  of  expert  systems  too, 
are  seen  as  an  obstacle  [Qanccy8l,Brcukcr88a].  Should  one  succeed  in  hierarchically  structuring 
a  knowledgebase  in  a  useful  way.  then  the  explanation  could  also  be  improved.  Proper 
explanation  facilities  are  especially  important  in  expert  systems  that  suffer  from  an  evaluation 
problem.  If  no  satisfactory,  objective  evaluation  entity  can  be  found,  a  good  explanation  is 
indispensable. 

The  expert  system  software  crisis  is  not  ya  solved  if  a  solution  would  be  found  to  these 
subproblems:  a  coherent,  generaUy  t^Iicable  methodology  is  called  for,  as  is  a  uniform  and 
generally  applicable  specification  language.  These,  however,  are  aims  that  can  only  be  realized 
completely  when  these  subproblems  have  been  solved.  Furthermore,  it  is  doubtful  whether  an 
appropriate,  unifonn  methodology  can  be  found. 
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Because  in  practice,  the  label  expert  system  is  put  on  significantly  different  programs  and  every 
definition  encountered  in  the  respective  literature  leaves  room  for  further  inteipretation,  the 
validity  of  many  claims  about  expert  systems  in  general  becomes  quite  debatable.  Knowledge 
acquisition  as  a  bottle-neck  in  the  develojMnent  of  expert  systems,  for  instance,  has  recently  been 
contested  by  [Cullen88].  Qaims  about  quality  assurance  are  no  exception  in  this  respect.  To 
prevent  claims  that  were  legitimately  made,  for  example,  about  MYCIN  (or  with  MYCIN  in 
mind),  from  being  generalized  without  second  thought  into  the  realm  of  expert  systems  in  general, 
the  recognized  problems  must  be  coupled  with  the  characteristics  that  are  necessary  or  sufficient 
to  make  these  problems  appear. 

Figure  2. 1  takes  a  first  step  in  this  respect.  The  diversity  of  expert  systems  is  made  explicit  in  the 
form  of  ten  dimensions  in  which  individual  expert  systems  can  differ  and  that  are  important  with 
respect  to  the  amount  of  problems  with  which  one  will  be  confronted  during  the  development  of  a 
specific  expert  system. 

These  dimensions  are  subdivided  into  three  types;  domain-dimensions,  strucnire-dimensions  and 
futKtional  dimensions.  The  first  two  relate  to  the  field  of  knowledge  and  what  may  be  expected 
from  the  expert  system  in  absolute  terms.  Characterizations  of  these  dimensions  must  therefore  be 
classified  as  preconditions.  As  far  as  functional  dimensions  are  coiKemed,  domain  and  global 
specifications  are,  within  certain  limits,  controlling  factors. 

Structure-dimensions  relate  to  the  structure  of  knowledgebases,  the  knowledge  representation  so 
to  speak.  The  characterization  of  an  expert  system  in  terms  of  these  dimensions  is  mairdy  a  matter 
of  choice  for  the  designers. 

A  number  of  these  dimensions  deserve  further  discussioa  Conceptual  complexity,  the  number  of 
object  and  relation  types  (predicates)  is  a  consideration.  Knowledge  extmsiveness  refers  to  the 
(average)  number  of  specific  objects  or  relations  per  type,  in  other  words,  the  average  number  of 
instances. 

Hierarchy  and  modularity  are  seen  as  separate  dimensions  because  systematic  subdivision  of  a 
knowledgebase  (hierarchy)  does  not  guarantee  that  changes  in  one  of  the  ccmiponent  parts  will  not 
yield  unexpected  effects  concerning  the  functionality  of  the  other  pans  (modularity). 
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Dimension 

Type 

Danger  zone 

Associated  problems 

Controllability  of  the 
final  result 

1  domain 

low 

evaluation 

explanation 

Accessability  of 
knowledge 

domain 

low 

knowledge  acquisition 
explanation 

Consensus  about 
knowledge 

domain 

low 

knowledge  acquisition 

evaluation 

explanation 

Conceptual  complexity 

domain 

high 

knowledge  acqpjisition 

Knowledge  extensiveness 

domain 

high 

knowledge  acquisition 
testing 

Hierarchy 

structure 

low 

testing 
maintenance 
explanation 
knowledge  acquisition 

Modularity 

structure 

low 

testing 

maintenance 

Uncertain  inference 

functionality 

high 

knowledge  acquisition 
testing 

Expert  emulation 

functionality 

high 

knowledge  accjuisition 
explanation 

Creativity 

functionality 

high 

i 

knowledge  acquisition 

Figure  2.1:  Dimensions  of  an  expert  system  in  relation  to  quality  assurance. 


Uncertain  inference  implies  the  use  of  certainty  factors  or  other  confidence  factors  as  well  as  the 
use  of  heuristic  rules. 

Creative  systems  refer  to  design  systems  that  are  not  based  on  decision  trees,  in  other  words,  that 
find  other  ways  to  a  solution  than  making  a  selection  from  a  limited  number  of  alternatives. 
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2.3  A  quality  framewoik 

Unnecessary  to  say  that  claims  about  the  quality  of  software  are  only  possible  when  the  notion 
itself  has  been  properly  defined  and  can  be  measured.  Whether  a  piece  of  software  has  a  good 
quality  is  difficult  to  establish;  the  absence  of  quality,  however,  is  much  easier  to  demonstrate. 
This  section  intends  to  build  a  framework  in  which  the  quality  assurance  of  expert  systems  can 
take  place.  Three  aspects  are  important; 

•  analysing  a  (object)  system  which  then  wiU  result  in  system  specifications; 

•  development  process  of  expert  systems; 

•  expert  system  as  a  product 

The  relationship  between  these  three  aspects  can  be  represented  as  follows: 
product  =  f( development  process  (system  specifications)}. 

Applying  the  development  process  to  the  specifications  results  in  the  expert  system  as  a  product. 
Quality  assurance  thus  takes  place  in  three  ways; 

•  validation  and  verification  of  system  specifications; 

•  structured  development  process; 

•  testing  and  evaluating  the  product 

Because  the  system  specifications  are  of  great  importance  during  the  whole  development  phase  of 
a  system  and  many  errors  are  only  detected  after  implementation,  in  this  study  special  attention 
will  be  focused  on  quality  assurance.  During  validation  of  system  specifications  it  is  determined 
whether  these  are  in  accordance  with  the  user  needs.  During  the  validation  process,  the 
specification  is  confronted  with  reality.  During  verification  the  specification  is  confronted  with 
the  implementation. 

The  need  for  a  structured  development  process  has  long  since  been  recognized.  Phasing 
methodologies  such  as  System  Development  Methodology  (SDM)  play  an  important  role  in  this 
respect  [Turners?].  In  the  field  of  expert  systems  development,  a  mediodology  is  used  that  was 
derived  ftom  SDM,  namely  Structured  Knowledge  Engineering  (SKE)  [SKE88].  For  specific 
activities  in  expert  system  development,  e.g.  knowledge  acquisition,  this  methodology  provides 
detailed  guidelines.  During  the  various  stages  of  develqxnent,  (sub)systems  must  be  tested  with 
regard  to  functionality  and  user  needs.  Testing  can  be  considered  as  observing  the  bdiaviour  of  a 
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(sub)systein  when  it  is  fed  with  a  controlled  selection  of  data.  During  evaluation,  experiments  are 
carried  out  after  which  the  results  are  compared  with  the  solutions  for  a  specific  problem  given  by 
a  recognized  expert. 


2.4  Quality  criteria 

Llinas  and  Rizzi  [Llinas87]  present  the  following  list  with  lest  and  evaluation  criteria: 

•  ctmsistency  and  Completeness  of  the  knowledgebase; 

•  software  quality  criteria; 

•  man-machine  interface; 

•  costs; 

•  quality  of  explanation; 

•  relevance; 

•  reliability; 

•  portability; 

•  contribution  to  organizational  aims; 

•  quality  and  accuracy  of  decisions,  advice  or  recommendations; 

•  correctness  of  reasoning; 

•  system  efficiency; 

•  adaptability  (maintainability); 

•  lun-time; 

•  computational  efficiency; 

•  intelligent  behaviour. 

This  list,  however,  is  not  a  good  starting  point,  because  it  contains  too  many  vague  and 
overi!4)ping  items.  Moreover,  some  of  the  criteria  (consistency  and  completeness,  run-time)  fall 
within  the  scope  of  units  of  measurement,  whereas  others  (software  qualiQr  criteria,  costs, 
contribution  to  organizational  aims)  provide  a  very  trivial  elaboration  of  the  philosophies 
associated  with  them. 

Subject  of  this  study  is  the  quality  assuraiKe  and  especially  the  methods  and  techniques  thru  can 
lead  to  a  product  of  better  quality.  Figures  2.2,  2.3  and  2.4  present  a  vision  cm  quality,  (^ality 
aspects  of  specificatitm,  process  and  product  are  givoi  separately,  for  easy  interpretation  of  the 
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used  tems.  Nevertheless  some  terms  deserve  a  bit  of  explanation.  The  terms  consistency  and 
completeness  are  used  in  the  literature  for  quite  diifemit  notions,  depending  on  context. 


I  usefulness 

I  I  reliability 

I  I  I  robustness  (x) 

I  I  I  controllability  (x) 

I  1  efficiency 

I  I  I  task  division  (x) 

I  I  I  task  allocation 

II  I  I  capability 

II  I  I  portability  (x) 


Figure  2.2;  Quality  criteria  for  the  develofHnent  process. 


Falsifiability  of  specifications  refers  to  the  fact  that  specifications  that  are  vague  to  such  an  extern 
that  even  very  diverse  implementations  can  be  verified  with  them,  are  rather  useless.  In  the  case 
of  robustness  of  the  developing  process,  one  must  think  of  escape  clauses  that  anticipate 
unforeseen  set-backs  and  absorb  these  as  good  as  possible.  The  robustness  of  a  product  refers  to 
the  protection  against  unexpected  external  influences.  These  can  be  of  a  human  or  technical 
nature.  Fail-safe  refers  to  responding  to  technical  defects  such  as  power  failure,  whereas  monkey- 
proof  ^lies  to  human  misbehaviour  behind  the  keyboard. 

Especially  expert  systems  evoke  a  call  for  the  specification  at  knowledge  level 
[Newell823miker87]  at  a  large  disumce  from  the  implemerttatitm.  This  in  itself  may  be  very 
advantageous,  provided  that  this  distance  can  be  bridged  in  a  formalized  manner  through  a 
(number  oO  spedficationfs),  which  are  much  closer  to  the  implementatitMi. 
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usefulness 
1  reliability 

I  I  integrity 

I  I  I  consistency  (x) 

I  I  I  completeness  (x) 

t  1  testability 

I  I  I  modularity  (x) 

I  I  I  falsifiability  (x) 

I  I  efficiency 

I  I  t  ease  of  design 

I  I  I  ease  of  development 

adaptability 
I  maintainability 

i  I  readability 

I  I  modularity 

I  portability 

I  I  hardware  independent 
I  I  implementation  independent 


Figure  2.3:  Quality  criteria  for  the  specification. 


The  layout  of  the  diagrams  indicates  the  hierarchy  of  the  concepts.  Usefulness  and  adaptability, 
for  instance,  are  seen  as  the  most  important  quality  aspects  of  specifications,  with  usefulness 
being  subdivided  into  relial^ity  and  efficiency.  For  a  better  understanding,  one  must  realize  that 
some  (actually  most)  of  the  criteria  in  the  diagram  and  entities  are  desiderata  of  the  specification, 
the  process  or  the  fHodua.  whereas  others  are  desiderata  of  the  techniques  that  ate  to  be  af^lied. 
The  latter  category,  for  instance,  includes  ease  of  design  and  development  in  Reifications, 
(^ality  aspects  tlm  will  receive  most  attention  in  dus  study  are  marked  with  an  (x).  A  more 
detailed  explanation  of  the  selection  of  just  these  aspects  as  key  notions  of  the  quality  of 
specification  uid  produa.  will  be  given  in  the  respective  chapters. 
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I  usefulness 
I  I  reliability 

I  I  I  integrity 

II  I  I  consistency  (x) 

II  I  I  completeness  (x) 

I  I  I  correctness 

I  I  I  robustness 

II  I  I  fail-safe  and  monkey-proof 

II  I  I  graceful  degradation 

I  I  I  testability 

II  I  I  modularity  (x) 

II  I  I  closely  related  to  specincation(x) 
I  I  efficiency 

I  I  I  computational  efficiency 

I  I  I  user  friendliness 

I  adaptability 
I  I  maintainability 

I  I  I  readability 

I  I  I  modularity  (x) 

I  portability 

I  I  hardware  independent 

I  integratability  (connectivity  to  other  software) 

Figure  2.4:  (^ality  criteria  for  the  product. 
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3  THE  LIFE-CYCLE  OF  AN  EXPERT  SYSTEM 

3.1  Intioduction 

Practice  reveals  that  the  development  problems  of  many  expert  systems  can  essentially  be 
reduced  to  management  problems  [CuUenSS].  This  seems  to  be  trivial  and  certainly  is  not  specific 
to  the  development  of  expert  systems,  yet  during  development  this  tends  to  be  forgotten.  As  such, 
in  practice  it  may  turn  out  to  be  the  largest  bottle-neck.  Serious  quality  assurance  of  an  expert 
system  must  insist  on  management  support  of  the  product  under  development,  and  phasing  must 
include  enough  room  for  a  feasibility  study  and  the  establishment  of  an  evaluation  iramewoilc  in 
an  early  stage.  This  chapter  deals  with  a  number  of  project  phasing  methodologies  that  can  act  as 
guidelines  during  the  development  of  an  expert  system. 


3.2  A  conventional  look  at  an  unconventional  development 

Most  development  methodologies  for  expert  systems  emphasize  the  knowledge  acquisition  phase. 
This  is  no  wonder,  when  we  consider  the  degree  of  complexity,  intrinsic  to  this  activity.  However, 
the  development  of  an  expert  system  is  not  limited  to  this  phase  only.  Especially  the  Structured 
Knowledge  Engineering  methodology  (SKE)  [SKE88]  makes  a  (concise)  statement,  based  on  the 
System  Development  Methodology  (SDM)  [Tumer87],  about  the  arrangement  of  the  other 
phases,  such  as  technical  design,  implementation,  testing,  acceptance  and  maintenance.  The 
advantage  of  the  SKE  approach  is  that  it  embraces  a  familiar,  conventional  method  of 
development  improves  the  chance  of  actual  application. 

SDM  is  a  phasing  methodology  for  the  management  of  system  development  projects.  SDM  deals 
with  the  whole  of  a  system's  life-cycle,  from  information  planning  through  to  maintenance.  The 
SDM  manual  is  a  guidelitte  for  managing  projects,  arranging  the  documoitation,  development  and 
management  of  information  systems.  For  every  development  phase,  the  activities  to  be  performed 
are  irKluded.  For  every  activity  are  regi^ered:  a  description,  one  or  more  analysis  and  design 
methods  (e.g.  NIAM,  JSD  or  SA).  a  number  of  deliverables  or  documents  and  literature 
references. 
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This  manual  fulfils  the  role  of  a  cookbook;  not  all  of  the  details  are  important  to  every  project.  By 
considering  it  as  an  extensive  check-list,  it  is  possible  to  make  plans,  guide  development  team 
members  through  their  tasks  and  establish  standards  for  documentation.  Special  attention  is  paid 
to  human  interaction  with  the  system,  which  benefits  the  development  of  an  adequate  man- 
machine  interface. 

SDM  consists  of  the  following  phases: 

•  information  plarming:  the  development  of  an  information  plan  in  accordance  with  the 
aims  of  the  organization; 

•  defmition  study:  defining  the  system  specifications,  choosing  a  method  to  develop  the 
system  and  making  an  initial  cost-benefit  analysis; 

•  basic  design:  refitting  the  system  reifications  and  a  first  system  design; 

•  detailed  design:  designing  subsystems  and  their  components; 

•  implemenution:  acquisition  of  devices  and  software,  after  which  the  system  will  be 
programmed  and  tested; 

•  installation  and  conversion:  transition  to,  and  utilization  of  a  new  system; 

•  use  and  management:  maintaining  the  system  in  such  a  way  that  it  meets  the 
specifications  that  evolve  from  the  organisation. 

The  acceptance  and  integration  phases  will  not  reaUy  differ  from  their  conventional  system 
development  counterparts.  This  does  in  no  way  imply  that  these  phases  would  be  of  lesser 
importance.  Emphasis  must  go  to  the  phases  regarding  knowledge  and  information  requirement 
analysis  up  to  the  implementation  of  the  expert  system. 

The  first,  more  or  less  structured  development  cycle  for  expert  systems  has  been  described  by 
Shortliffe  and  Davis  [Shortliffe7S].  This  cycle  is  ktwwn  under  the  name  of  SIGART  development 
cycle  because  it  was  mentioned  for  the  first  time  in  the  SIGART  Newsletter  no.  55. 

The  SIGART  development  cycle  has  the  foUowing  phases: 

1 .  top-level  design:  definition  of  long-term  objectives; 

2.  implementatitHi  of  a  Mark  I  prototype  to  demonstrate  its  feasibility; 
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3.  reiineinent  of  the  system,  usually  by: 

a.  lunning  informal  test  cases  to  elicit  feedback  from  the  expert,  which  results  in  a 
refined  Mark  II  prototype; 

b.  making  Mark  II  available  to  'friendly'  users; 

c.  revising  the  system  on  the  basis  of  user  feedback; 

d.  making  the  revised  Mark  II  avaUable  to  users  and  return  to  phase  3b; 

4.  structured  evaluation  of  performance; 

5.  structured  evaluation  of  user  acceptability; 

6.  operational  use  in  a  prototype  environment  for  an  extetxled  period; 

7.  making  follow-up  studies  to  investigate  whether  the  system  is  ready  for  large  scale 
operation; 

8.  adapting  the  program  so  that  it  is  suitable  for  a  wide  distribution; 

9.  general  release  and  marketing  with  cleaily-defmed  plans  for  maintenance  and  updating. 

The  SIGART  life-cycle  further  consists  of  numerous  recommendations,  but  does  not  contain  any 
further  methods. 

Llinas  and  Rizzi  [LlinasS?]  state  that  this  phasing  is  essentially  a  r^id  prototyping  iq>proach. 
They  even  go  further  they  believe  that  the  development  process  of  (non-trivial)  expert  systems  is, 
by  necessity,  of  an  evolutionary  character,  as  long  as  there  is  no  better  understanding  of  the 
knowledge  engineering  process.  They  draw  a  parallel  with  the  early  days  of  the  development  of 
FORTRAN,  before  the  days  of  programming  paradigms  when  program  development  was 
performed  in  exactly  the  same  way  as  in  the  SIGART  development  cycle. 


3.3  Rapid  prototyping 

As  we  have  already  mentioned  in  the  previous  section,  r^d  prototyping  is  a  very  important 
paradigm  in  the  development  of  expert  systems.  Rapid  prototyping  does  not  stem  from  the  field  of 
AI.  It  originates  from  conventional  software  engineering  as  a  remedy  against  the  language  barrier 
that  frequently  shows  up  in  the  conversation  between  end  users  atxl  system  developers.  Whatever 
the  value  of  standardized  diagrams  aid  textual  Reifications  for  system  development  and  the 
associated  quality  assursmee,  they  appeared  unsdde  to  offer  the  end  user  sufficient  information  (or 
elicit  enough  information  for  that  matter)  to  guarantee  the  everttual  user  satisfactitm  regarding  the 
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implemenution.  This  phenomenon  can  be  partly  traced  back  to  the  end  user  misunderstanding 
symbols  that  are  familiar  to  the  software  specialist;  for  another  pan  it  can  be  reduced  to  the  simple 
fact  that  it  is  difficult  to  estimate  just  how  convenient  a  certain  routine  really  is  without  actually 
trying  it.  A  prototype  offers  this  possibility,  as  a  (simplified)  version  of  the  system  that  is  to  be 
developed. 

When  rapid  prototyping  appeared  successful  in  finding  suitable  user  interfaces,  other  applications 
too,  were  subjected  to  it.  Hayward[Hayward87]  identifies  five  kinds  of  prototyping; 

•  experimental  (for  finding  a  suitable  user  interface  or  solution  strategy); 

•  exploratory  (for  requirements  analysis); 

•  performance  (for  predicting  response  times,  use  of  memory  etc.); 

•  organisation  (for  predicting  the  effects  on  the  organisation); 

•  evolutionary  (for  complete  system  development). 

Rapid  prototyping  is  only  feasible  if  the  development  of  the  prototype  can  be  accomplished 
rapidly  (inexpensively).  This  was  only  possible  after  fourth-generation  languages  (application 
generators)  became  available.  The  fact  that  an  expert  system  shell  can  in  fact  be  used  as  an 
application  generator,  accounts  for  the  success  of  these  shells  as  well  as  for  the  advance  of  r^id 
prototyping  as  a  development  method  for  expert  systems.  The  knowledgebase  itself,  is  then  often 
labelled  as  a  specification  of  the  system.  Doyle  [DoyleSS]  remarks  that  'with  the  AI  approach  (...), 
the  interpreted  specification  acts  as  an  inexpensive,  but  full-grown  prototype  (...)  so  that  this 
approach  easily  overshadows  others,  when  formulating  the  problem  becomes  complicated  -  which 
usually  is  the  case'.  Not  everyone  is  (nowadays)  biased  so  positively  towards  prototyping  of 
expert  systems.  Breuker  and  Wielinga  [BieukerSSa],  for  instatKe,  referring  to  [Hayes-Roth831, 
establish  that  'rapid  prototyping  soon  leads  to  backtracking  and  discarding  systems:  in  any  case,  it 
does  not  lead  to  a  properly  manageable  development  process'. 

The  lack  of  concensus  between  these  experts  can  partly  be  explained  from  the  fact  that  notitms  get 
confused:  althou^  piototyping  is  a  very  valid  and  reliable  aid  within  a  structured  development 
process  without  endangering  the  controllability,  the  way  in  whidi  (evolutionary)  prototyping  in 
expert  system  development  is  done,  can  easily  give  rise  \o  a  chaotic  life-cycle  and  likewise,  a 
chaotic  produa.  This  is  probtdtly  also  due  to  the  popularisation  of  expert  systems  over  the  past 
decade.  Every  experienced  software  specialist  should  know  that  every  program  that  was  subjected 
to  numerous  dianges  should  be  te-«ialysed.  oftmt  be  re^ctuied  and  should  sometimes  be 
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completely  rewritten.  Therefore,  it  is  no  wonder  that  an  'incremental  evolution'  is  reg&rded  as  a 
risky  woridng-method  [HaywardS?]. 

Doyle  too,  stresses  that  the  merits  of  expert  system  shells  should  not  be  looked  for  in  the  fuial 
product  -  'that  would  probably  be  more  efficient  when  written  in  a  procedural  language,  such  as  C 
-,  but  rather  in  the  fact  that  a  shell  enables  rapid  prototyping  and  thus  substantially  reduces  costs 
[Doyle85].  As  an  example,  he  gives  the  PUFF  system  for  diagnosing  breathing  difficulties.  Its 
development  took  a  lot  of  testing  and  reformulating.  Yet,  less  than  SO  hours  of  expert  consultation 
and  less  than  10  man  weeks  of  programming  were  needed  to  obtain  a  satisfactory  system  with 
some  fifty  production  rules. 


3.4  Weitzel-Kerschberg  life-cycle  model 

It  is  expected  that  a  development  method  for  expert  systems  incorporates  a  good  phasing,  leads  to 
clearly  defined  intermediate  and  final  products,  can  be  applied  iteratively,  allows  for  prototyping 
and  provides  a  well-defined  testing  and  evaluation  phase.  Though  this  list  is  nuher  ambitious, 
Weitzel-Kerschberg's  method  appears  a  good  candidate  [Weitzel89].  Important  is  that  this  author 
sees  'systems  based  on  knowledge'  as  an  evolutionary  development  of  conventional  systems.  It  is 
no  wonder  therefore,  that  he  is  a  staunch  proponent  of  so-caUed  'expert  database  systems'  that  are 
a  fusion  of  database  and  expert  systems. 

This  development  method  relies  considerably  upon  techniques  such  as  prototyping,  without 
ignoring  the  development  of  a  conceptual  model  of  the  pnoblem  domain.  In  order  not  to  become 
stuck  in  sequential  development,  the  individual  phases  of  this  method  consist  of  'processes'  that 
can  be  activated,  deactivated  and  reactivated.  A  precondition  is  that  a  pixxress  can  only  be 
activated  when  the  previous  process  has  been  activated  at  least  once  (with,  of  course,  the 
exception  of  the  very  first  process). 

A  great  deal  of  attention  is  paid  to  activities  such  as  refining,  redefining  and  restructuring.  The 
first  process,  problem  identification,  is  cotKemed  with  an  initial  survey  of  the  matter  on  hand. 


Next,  a  knowledge  and  information  requirements  analysis  is  performed  within  the  problem 
deration  process.  In  close  cooperatitm  with  an  expert,  an  informal  characterization  is  made  of  the 
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notions  and  concepts  that  play  a  role  in  the  problem  area.  The  question  what  purpose  the  system 
to  be  developed  should  serve  is  answered  here  (interpretation  of  satellite  data,  displaying  data  in  a 
head-up  display  of  an  aircraft,  archiving  very  scarce  knowledge).  This  phase  is  also  meant  to 
judge  the  feasibility  of  the  project.  This  will  usually  be  done  by  interpreting  technical  and 
economical  factors.  It  is  expected  that  this  process  will  stand  a  good  chance  of  being  reactivated 
later  on,  especially  if  additional  data  are  found. 

The  process  of  subproblem  identification  subdivides  the  problem  into  manageable  parts.  This 
leads  to  a  further  reduction  of  the  risks  involved  because  now  it  becomes  possible  to  develop  a 
small  prototype  in  a  later  stage  for  every  subproblem,  that  will  afterwards  be  reintegrated. 

Concept  identification  and  definition  record  the  various  concepts  and  their  mutual  relationships 
that  are  of  importance  in  solving  a  problem.  This  process  implies  amongst  others  that  all  objects, 
attributes,  relationships  and  constraints  wiU  be  modelled.  foundation  is  laid  for  a 
knowledgebase;  what  happens  here  is  the  establishment,  of  a  conceptual  model.  Knowledge 
acquisition  techniques  too,  find  their  place  in  Uiis  process. 

In  the  conceptual  design,  logical  choices  are  made  concerning  knowledge  representation  and 
databases.  The  idea  that  an  Al-system  adds  to  the  value  of  a  conventional  system  is  true:  the 
knowledgebase  is  partitioned  into  data-based  and  rule-based  components. 

Detailed  design  defines  the  physical  system  design.  Here  for  instance,  the  descriptions  and 
pseudo-code  of  programs  are  produced.  This  boils  down  to  mapping  the  most  important  concepts, 
relationships,  information  flows  and  solution  strategies  within  a  formal  representation  framework. 
This  formalization-stage  eventually  results  in  a  prototype  that  is  built  with  the  use  of  a 
programming  language  or  an  expert  system  shell. 

Actual  production  of  the  prototype  takes  in  the  programming  process.  Here,  the  corrceptual 
design  is  converted  into  a  runnable  equivalent.  Regular  feedback  with  the  previous  process  can  be 
expected. 

The  process  of  testing  the  knowledge  and  reasoning  capabilities  comes  next  Working  closely 
together  with  the  expert,  the  system  is  tested.  It  is  attempted  to  correct  evident  bugs. 
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Finally,  the  validation  process.  The  system  will  ik)w  be  confronted  with  a  great  number  of  cases 
concerning  the  relevant  problem  area.  System  performance  is  then  compared  to  that  of  the  human 
expert. 

The  Weiuel-Kerschberg  method  incorporates  the  possibility  to  integrate  the  development  of 
expert  systems  into  a  larger,  comprehensive  system.  The  suggested  approach  can  be  considered  as 
a  separate  phase  within  an  SDM-like  phasing  methodology.  At  this  point  conventional  and 
unconventional  system  development  go  hand  in  hand. 
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4  KNOWLEDGEBASE  SPECIFICATION  METHODOLOGIES 

4.1  Introduction 

Purpose  of  this  chapter  is  the  introduction  of  a  method  to  consistently  and  completely  record  the 
specifications  of  an  expert  system.  The  mani^r  in  which  a  certain  knowledge  domain  of  an  expert 
system  is  specified  does  not  really  differ  from  the  way  it  is  done  in  the  development  of  a  database 
system.  This  means  that  the  knowledgebase,  being  the  central  pan  of  an  expen  system,  is  a 
replication  of  a  knowledge  domain  in  terms  of  a  model.  The  semantics  of  the  knowledge 
contained  in  the  knowledgebase  must  be  specified  unambiguously.  A  conceptual  scheme  of  the 
knowledge  is  designed,  a  semantic  definition. 

Because  conventional  systems  wiU  eventually  make  use  of  knowledge  modules,  it  is  interesting  to 
take  a  look  first  at  the  possibilities  that  conventional  methods  for  analysis  and  design  offer  to 
specify  these  modules.  This  approach  makes  integration  easier.  After  that,  a  description  follows  of 
a  method  specifically  developed  for  the  specification  of  the  knowledgebase  of  an  expert  system. 
The  chapter  will  be  concluded  with  the  description  of  an  extended  conventional  method  that  looks 
very  promising  as  far  as  quality  assurance  of  expert  systems  is  concerned. 


4 . 2  Conventional  sped  fication  methodologies 

System  development  should  not  be  an  art.  A  number  of  conditions  can  be  attached  to  the  methods 

and  techniques  that  are  used  for  system  analysis  and  design.  They  should: 

•  be  independent  from  the  analyst/designer’s  inventiveness; 

•  contribute  to  proper  controllability,  through  division  into  phases  with  well-defined 
products; 

•  be  rational,  based  on  fixed  prindples  so  that  they  contribute  to  objective  dedsion- 
making; 

•  support  an  iterative  develofxnent  process; 

•  be  transferable,  so  that  design  decisions  are  identifiable  and  reprodudble;  there  is 
uniformity  in  the  individual  solutions  and  they  are  accessible  by  means  of  proper 
documentation; 

•  be  practical  (or  else  remain  unused); 
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•  guarantee  correctness  through  design  and  not  through  testing; 

•  be  independent  from  the  implementation  environment  and 

•  be  supported  by  computer  aided  software  engineering  (CASE)-tools. 

Most  analysis  and  design  methods  can  be  subdivided  into  a  number  of  categories: 

•  process-oriented; 

•  data-oriented; 

•  objea-oriented. 

In  a  process-oriented  method,  processes  are  analysed  first  after  which  follow  the  data  and  data 
structures,  necessary  for  effectively  executing  the  processes  described.  The  product  is  a 
description  of  the  possible  processes  that  alter  the  states  in  which  a  system  can  find  itself. 

A  data-oriented  method  gives  first  priority  to  analysing  the  data  and  data  structures  and  then 
proceeds  with  analysing  processes.  The  produa  is  a  description  of  the  possible  states,  state 
transitions  and  the  description  of  these  transitions. 

Object-oriented  methods  aim  at  dividing  the  state  description  into  elementary  state  descriptions 
(entities  or  objects).  These  entities  are  obtained  by  describing  the  possible  sequences  of  state 
transitions.  Added  to  that  is  a  description  of  the  dependencies  that  exist  between  the  state 
transitions  of  the  various  entities. 


4.2.1  Structured  Analysis 

Structured  Analysis  (SA)  is  a  process-oriented  method  of  analysis  [Gane78].  The  goal  of  SA  is 
the  development  of  a  logical  system  model,  incorporating  graphical  techniques  that  support  the 
communication  between  users,  analysts  and  designers.  First,  the  proldem  area  is  modelled  by 
means  of  dataflow  diagrams.  These  consist  of  four  elements:  information  sources/sinks,  data- 
transforming  processes,  dataflows  and  data  storage.  Diagrams  are  ordered  hierarchically,  which 
means  that  a  process  itself  can  in  its  turn  be  specified  in  the  form  of  a  dataflow  diagram  (DFD). 

While  making  DFDs,  names  for  processes,  dtfaflows,  data  storage  and  data  sources  are  stored 
into  a  so-called  dau  dictionary,  which  may  be  seen  as  a  set  of  definitions  at  a  logical  level. 
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Next,  it  must  be  recorded  what  goes  on  in  the  processes.  In  other  words,  what  functions  must  be 
executed  by  the  system.  This  can  be  done  with  the  help  of  decision  trees  that  represent  what 
function  must  be  executed  at  what  moment,  or  so-called  'structured  English',  which  describes  the 
operation  of  a  function  in  words. 

After  that  the  data  storage  is  analysed.  By  using  techniques  like  normalization  and  bubble 
charting,  it  is  determined  how  data  are  stored  in  a  database  and  what  access  paths  are  contained  in 
functions. 

An  iterative  process  records  the  logical  DFDs,  the  data  dictionary,  the  process  description  and  the 
definition  of  the  database,  that  all  together  form  the  logical  speciHcation  of  the  system. 


4.2.2  Nijssens  Information  Analysis  Method 

Nijssens  Information  Analysis  Methodology  (NIAM)  is  an  example  of  a  data-oriented  method 
[WinhaeckenSS].  NIAM  uses  three  notions: 

•  knowledge  is  everything  someone  knows  about  a  certain  subjea  or  a  certain  area; 

•  information  is  knowledge  that  can  be  transfered; 

•  communication  is  the  exchange  or  transfer  of  information. 

That  part  of  the  real  or  abstract  world  to  which  the  information  exchanged  through 
communication  relates  is  called  reality  or  Urtiverse  of  Discourse.  When  we  limit  ourselves  to 
commurticative  processes  that  use  an  information  processing  system,  then  the  information  has  to 
be  represented  in  the  communication  medium;  these  are  the  data.  Commurucation  between 
humans  using  an  information  processing  system  is  only  practical  if  it  is  defined  which  data  may 
be  used  for  the  commurucation,  and  what  the  semantics  of  this  data  is.  These  matters  are 
described  in  a  so-called  omceptual  grammar.  The  information  analysis  process  may  now  be  seen 
as  an  activity  'where  information,  exchanged  during  commurucation  processes,  is  analysed  and  a 
(conceptual)  grammar  for  these  communication  processes  is  formulated,  based  on  this  analysis' 
[WintraeckenSS]. 

In  this  way,  NIAM  offers  the  possibilities  to  define  the  specifications  in  a  formal  manner.  The 
analysis  phase  yields  a  conceptual  grammar  that  is  transformed  during  the  design  and 
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implementation  {4iase  into  a  computer  based  information  system.  The  conceptual  grammar 
describes  three  aspects: 

•  communication,  or  information  flows; 

•  transformation,  or  functions  that  influence  data  elements; 

•  logical  data  storage  method. 

Application  principles  of  NIAM  are; 

•  communication  takes  place  by  means  of  natural  language  sentences; 

•  communication,  transformation  as  weU  as  the  data  storage  method  can  be  described  by 
means  of  a  conceptual  grammar. 

NIAM  is  based  upon  a  number  of  clear  concepts  with  which  specifications  can  be  described. 
System  development  along  the  lines  of  NIAM  consists  of  three  phases.  Firstly,  the  analysis  of  the 
activities.  This  aims  to  form  an  aaivity  model  and  to  determirte  the  user  needs.  With  the  help  of 
methods  and  techniques  gleaned  from  the  discipline  of  business  administration,  the  activities  can 
be  traced.  The  results  of  this  phase  are  usually  considered  as  data  during  the  initial  stages  of  the 
NIAM  analysis.  Secondly,  the  analysis  of  infonnation.  The  conceptual  grammar  must  be  defined 
during  this  phase.  Infonnation  flow  diagrams  must  be  made,  representing  the  relationships 
between  futKtions,  information  flows,  data  storage  and  information  sources/siiiks.  Top-down 
decomposition  leads  to  a  hierarchical  ordering  of  diagrams.  Every  flow  or  transformation  is 
described  by  means  of  natural  language.  After  that,  the  information  structure  of  the  flows  and  data 
storage  must  be  established.  The  result  is  an  information  structure  diagram.  The  consistency  of 
in^t  and  output  data  in  transfomiation  must  be  guaranteed.  Finally  it  is  established  what 
functions  must  be  automated  and  a  deflnition  of  the  user  interfaces  after  which  die  construction  of 
the  system  may  begin.  Lastly  follows  the  actual  construction  of  the  system.  Apfdicaticm  programs 
and  databases  will  now  be  implemented. 


4.2.3  Jackson  System  Development 

Jadcson  System  Development  (JSD)  is  an  object-oriented  specificatitm  method  that  spans  the 
whole  life-cycle  of  a  system  from  uialysis  throu^  to  imidementation  [SutclifficSS].  Aim  is  the 
develqxnent  of  reliable  systems  thtt  can  be  prc^ily  maintained  by  means  of  concise, 
unambiguous  and  (yet)  readable  specificttions.  This  method  differs  ftxim  other  structured 
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methods  such  as  Stnictuied  Analysis  because  it  emphasizes  modelling  and  takes  the  factor  time 
into  account. 

JSD  consists  of  three  {biases,  with  a  distinct  beginning  and  end.  Project  control  is  simplified  by 
the  definition  of  intermediate  products.  First,  there  is  a  modelling  phase.  A  description  of  reality 
is  made  by  recording  the  actions  that  are  performed  and  analysing  the  objects  subjected  to  these 
actions.  By  making  a  list  of  objects  and  actions,  a  definition  is  obtained  of  that  part  of  reality  that 
falls  within  the  limits  of  the  system.  Actions  performed  or  experienced  by  an  object  as  well  as  the 
order  in  which  this  must  happen,  are  structured.  In  this  way.  reality  is  described  in  terms  of 
actions  and  objects  within  a  process  model. 

The  second  phase  is  the  networking  phase.  The  i»evious  phase  distinguishes  one  or  more 
subsystems,  to  which,  in  the  networking  phase,  inputs  and  outputs  are  assigned.  The  output 
system  that  is  the  user  interface,  will  now  be  analysed  as  is  the  input  subsystem  that  takes  care  of 
validation.  Furthermore,  the  preconditions  are  specified  as  far  as  time  aspects  and  system 
processes  are  concerned. 

Finally,  the  implementation  phase.  The  logical  system  specification  established  so  far  can  be  seen 
as  a  network  of  parallel,  communicating  processes,  which  is  transformed  into  a  sequential  design 
using  a  'scheduling'  technique.  This  activity  is  followed  by  detailed  technical  design  and 
programming. 

JSD  starts  with  the  analysis  of  the  most  important  system  structures  to  form  a  model  of  the  reality 
on  the  basis  of  this.  Only  then  the  ftmctionality  of  the  system  will  be  added.  Design  decisions  of  a 
technical  nature  are  postponed  until  the  final  phases  of  system  development 

The  principles  on  which  JSD  is  based  are  the  following: 

•  before  a  system  can  be  built  it  must  be  understood; 

•  to  come  to  an  understanding,  dw  proUem  must  be  modelled; 

•  systems  are  always  concerned  with  the  transformation  of  things,  therefore,  the 
transformation  process  needs  to  be  modelled. 

The  emphasis  lies  on  the  deeper  structure  ctf  the  problem  rather  than  on  analysis  of  functions  diat 
can  be  easily  tfoserved,  as  is  done  in  methods  sudi  as  Structured  Arudysis.  In  JSD  it  is  importam 


to  know  how  objects  change  in  time.  If  the  way  in  which  these  changes  take  place  is  properly 
modeUed,  the  system  should  function  well.  The  importance  of  the  factor  time  is  evident  JSD  is 
iK)t  directly  concerned  with  modelling  the  relationships  between  data,  as  is  NIAM.  This  must  be 
derived  indirectly  from  the  conununication  process  model. 

In  object-oriented  modelling,  the  system’s  ability  to  represent  the  behaviour  of  objects  in  reality  is 
important.  A  resemblance  to  JSD  is  clearly  present.  Ariother  innovative  element  of  JSD  is  the 
view  that  a  system  can  be  considered  as  a  cluster  of  subsystems  that  exchange  messages.  If  a 
separate  system  process  must  be  created  for  each  subsystem,  it  is  strongly  reminiscent  of  new 
computer  architectures  where  each  process  has  its  own  processor  available,  that,  in  its  own  turn,  is 
cormected  to  other  processors.  The  specification  can  then  be  represented  in  hardware. 

4.3  AI  speci  fication  methodology 

4.3.1  Knowledge  Acquisition,  Documenution  and  Structuring 

The  adherents  of  Knowledge  Acquisition,  Documentatitxi  and  Structuring  (KADS)  [Breuker87] 
oppose  the  traditional  AI  approach  to  ktK)wledge  acquisition  prototyping.  Breuker  and  Wielinga 
maintain  that  prototyping,  because  it  forcibly  imposes  abstraction,  produces  an  unstructured 
knowledgebase  without  the  conceptual  insight  necessary  for  explanation  and  maintainability 
[BreukerSSa].  They  claim  that  tlus  holds  true  especially  for  knowledge  domains  of  non-trivial 
complexity  or  size. 

[Hayward87]  compares  the  use  of  prototyping  versus  the  use  of  KADS  with  'craft  versus 
industry'.  He  considers  the  experimental  development  of  a  new  type  of  software  as  a  kind  of 
ingenious  handicraft.  As  soon  as  this  tecimology  has  matured  and  commercial  production  has 
been  started,  then  the  software-manufacturing  point  of  view  is  mote  appropriate.  The  exploratory 
approach  must  then  be  abandoned  in  favour  of  a  mote  structured  qjproach  that  can  be  controlled 
better. 

The  innovative  angle  of  KADS  in  the  fact  that  it  iMomotes  choosing  an  epistemological 
framewtnk  as  a  first  stq>  in  knowledge  acquisition.  Subsequeiuly,  one  tries  to  {dace  the 
accumulated  knowledge  inside  this  framework.  The  frunewoik  chosen,  inteipr^on  model  in 
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KADS,  is  a  domain-independent  abstraction  of  domain-dependent  (but  implonentation- 
independent)  conceptual  models. 

Bieuker  and  Wielinga  name  a  number  of  knowledge  acquisition  shells  as  foreninners  of  KADS 
because  they  impose  an  interpretation  model,  such  as  Roget  [BennettSS],  MOLE  [Eshelman87] 
and  SALT  [Marcus87]  as  well  as  TEIRESIAS  [Davis82]  and  OPAL  [Musen87]  that  are  used  for 
MYCIN  and  ONCOCIN.  respectively.  But  at  the  point,  however,  where  these  knowledge 
acquisition  shells  force  the  knowledge  engirreer  to  use  a  single  interpretation  model  that  is  part  of 
the  shell,  KADS  offers  the  possibility  to  alter  the  interpretation  model  completely  or  in  part  if  it 
appears  that  it  is  not  (at  all)  satisfactory. 

For  this  purpose,  a  typology  of  generic  tasks  has  been  established.  A  genetic  task  is  a  (domain- 
independent)  abstraction  of  an  inference  process  that  may  be  seen  as  a  conceptual  entity.  Oancy's 
heuristic  classification  is  an  example  of  an  elementary,  generic  task  [Gancey8S].  KADS  task 
typology  sees  heuristic  classification  as  a  form  of  singular  diagnosis,  that  in  its  turn  is  a  form  of 
diagnosis  etc.  An  assumption  of  KADS  therefore  is  that  inference  processes  can  be  abstracted 
within  a  specific  domain  (i.e.  stripped  from  its  domain-specific  notions)  that  can  still  be  described 
clearly  enough  to  recognize  an  equivalent  as  such  in  another  domaia  The  subjective  impressions 
of  those  that  have  used  KADS  in  practice,  partially  support  this  but  more  substantial  indications 
are  lacking  [Breuker88a]. 

Ideally,  an  interpretation  model  consists  of  an  inference  structure  and  a  task  structure.  The  first 
defiiKS  what  the  primitive  inferetKCS  are  (in  terms  of  name,  inputs,  outputs  and  a  very  broad 
description).  The  task  structure  defmes  how  these  basic  inferoKes  should  be  linked  together  to 
execute  the  task  in  question.  Unfortunately,  KADS  has  failed  to  work  out  the  task  structures  of 
inteipretation  models.  The  task  structures  used  in  actual  iq)plications  appeared  to  be  too  diverse. 
In  point  of  fact  this  means  that  KADS  interpretation  models  purely  consist  of  inference  structures 
like  that  shown  in  figure  4.1,  with  ctmcise  textual  information  added.  Primarily,  they  have  a 
declarative  (non-procedural)  characta*.  An  interpretation  model  purely  indicates  what  type  of 
objects  (meta-classes)  function  as  input  or  output  of  the  ^nq>tiate  types  of  primitive  infeteiKes 
(knowledge  sources)  and  does  not  say  anything  about  the  order  in  which  these  knowledge  sources 
must  be  activated,  nor  the  amditions  under  which  this  happens. 
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The  universal  structure  that  KADS  imposes  on  all  conceptual  models  is  a  four-layer  organization. 
The  domain  layer  coittains  knowledge  about  objects  (concepts)  of  the  actual  domain  and  their 
interrelationships.  The  inference  layer  specifies  what  inferences  are  possible.  The  task  layer 
specifies  in  what  situation  which  inferences  should  be  made  in  what  order.  The  strategy  layer, 
finally,  contains  the  knowledge  that  is  necessary  to  switch  to  another  solution  strategy  if  the 
present  is  unsatisfactory.  Breuker  and  Wielinga  remark  that  this  layer  does  not  or  barely  present 
itself  in  reality  [BreukerSSa]. 


4.4  A  comparative  analysis 

Until  now,  no  satisfactory  specification  language  has  been  found  for  inference  structures  in  expert 
systems.  The  diagrams  used  in  KADS  as  a  representation  of  interpretation  models  are  too 
ambiguous  to  act  as  a  specification  language  and  so  cannot  effectively  bridge  the  gap  between  the 
raw  output  of  knowledge  elicitation  and  implementation.  However,  the  selection  of  a  suitaUe 
interpretation  model  as  well  as  the  transition  of  a  conceptual  model  to  an  implementation,  must  go 
smoothly  if  KADS  is  ever  to  be  proposed  as  a  commercially  applicable  development  method. 

The  most  important  component  of  the  NIAM  methodology  is  the  transparent  representation 
formalism  which  makes  it  possible  to  generate  procedures  and  files  from  the  specifications  by 
means  of  an  application  generator  [Nijssen89}.  This  is,  as  indicated  above,  not  yet  possible  uang 
KADS.  Especially  database  experts  suggest  regularly  to  use  the  relatitmal  database  paradigm  for 
expert  systems.  The  NIAM  data  models  can  be  useful  for  domain  structuring,  something  that  has 
not  been  worked  out  sufficiently  yet  in  KADS  [BreukerSSa].  NIAM,  as  described  in 
[WintraeckenSS],  concentrates  on  data  structures  aixl  their  integrity.  Less  attention  goes  to 
defining  procedural  krowledge.  The  potential  of  Structured  Analysis  [Gane79]  as  well  as  that  of 
single-model  knowledge  acquisititm  tools  like  MOLE  [Eshelman87]  and  CSRL  [Bylander86] 
merit,  beside  NIAM,  further  investigation  as  far  as  diis  point  is  concerned. 

The  question  in  how  far  methods  such  as  SA,  JSD  and  NIAM  are  suitaUe  for  specifying  a 
knowledgebase  cwnot  be  answered  compfeiely.  Until  now,  not  many  cases  are  known  where  one 
or  more  of  these  methods  have  been  apfdied  building  intelligent  systems.  However,  in  the 
research  literature  there  is  much  speculation  about  this  subject 
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[Vogler88]  states  that  the  knowledge  levels  of  KADS  are  relatively  easy  to  transpose  to  SA.  The 
domain  level  is  similar  to  the  data-dictionary  that  is  compiled,  the  inference  level  can  be 
compared  to  a  data  flow  diagram  and  the  task  level  resembles  the  definitions  of  an  SA  process 
structure.  An  advantage  of  this  approach  is  that  the  integration  is  far  easier  in  conventional  system 
development.  A  disadvanuge  is  that  the  SA  method  remains  very  process-oriented.  This  houses 
the  danger  that  a  specification  of  the  domain  level  is  far  more  difficult  to  realize.  And  it  is 
especially  this  level  that  is  essential  to  an  expert  system  and  has  been  ignored  in  the  past  for  too 
long. 

There  are  no  known  examples  (yet)  of  an  expert  system  developed  with  the  use  of  JSD.  JSD 
strongly  emphasizes  the  modelling  of  objects  in  reality  (as  far  as  they  lie  within  the  limits  of  the 
system  that  has  to  be  dn\;loped)  and  the  actions  to  which  they  are  subjected. 

NIAM  offers  CAcellent  possibilities  to  specify  the  domain  level  of  an  expert  system  consistently 
and  completely  [Bruin88].  This  method  that  is  dau-oriented,  contrary  to  SA  and  JSD,  is  used 
among  others  for  the  development  of  a  conceptual  model  for  database  systems.  As  said  earlier, 
MAM  allows  for  the  definition  of  so-called  constraints.  These  rules  restrict  the  occurrence  of 
certain  facts  in  an  automated  information  system.  Especially  [Nijssen86]  explores  the  hypothesis 
that  an  expert  system  is  nothing  but  an  automated  information  system  containing  certain  human 
expertise.  This  expertise  (or  knowledge)  is  only  a  set  of  interrelated  facts.  Facts  may  be  entered 
by  the  user  (as  the  result  of  a  query  or  during  the  creation  of  the  database).  It  is  also  possible  that 
they  are  derived  from  other  facts  that  are  already  present  in  the  daubase,  with  the  help  of 
derivation  rules.  Finally,  they  can  be  the  result  of  a  reasoning  process:  by  applying  inference  rules 
on  facts  already  present,  new  facts  can  be  inferred. 

[Breuker87]  makes  a  subdivision  into  domain,  inference,  task  arul  strategic  levels.  Each  of  these 
levels  takes  a  particular  look  at  kiwwledge:  the  domain  level  may  be  described  as  the  most 
concrete  because  here  objects  in  the  problem  domain  and  their  mutual  relationships  are  defined. 
The  strategic  level  approaches  the  problem  domain  far  more  abstractly.  Indeed,  the  KADS  theory 
at  this  point  is  so  abstract  that  this  level  is  never  used  in  (nactice.  The  intermediate  levels 
(infereiKC  and  task)  contain  the  dynamisn  of  the  expert  system  as  it  were.  The  inference  levd 
indicates  what  elementary  inferences  (specification,  abstraction  etc.)  can  be  performed  with  the 
(Ejects  uid  relationships  at  the  domain  level  The  task  level  groups  (and  activates)  inferences  in 
order  to  define  a  certain  tadc  (classification,  diagnosis  etc.). 
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Undoubtedly,  the  specification  of  the  domain  level  has  a  strong  resemblance  with  the 
specifications  that  are  defined  during  the  development  of  a  database  system.  Here  too,  a  data 
dictionary  must  be  built:  a  file  that  records  character  and  nature  of  each  entry  that  is  used 
(definition,  significaiKe,  [^ysical  implementation  and  relationships  with  other  entries).  The 
inference  level  can  be  seen  as  more  process-oriented.  By  indicating  what  exactly  is  the  input  and 
output  of  elementary  inference  steps  (processes),  a  functional  model  of  the  problem  domain  can 
be  made.  The  task  level  makes  the  model  dynamic,  because  with  the  help  of  process 
specifications  it  describes  what  action(s)  must  be  executed.  Because  of  the  minor  practical 
relevance,  the  strategic  level  will  not  be  dealt  with  any  further. 

The  constraints,  that  can  exist  in  the  form  of  derivation  rules  or  inference  rules,  are  of  great 
importance.  By  choosing  a  method  that  offers  a  good  support,  not  only  at  the  domain  level  but 
also  at  the  more  abstract  levels  of  knowledge,  the  experience  that  was  accumulated  in  the 
database  community  with  regard  to  the  integrity  problem  can  be  transposed  to  the  knowledgebase 
community. 

The  next  paragraph  presents  an  extended  version  of  NIAM  (ExtendedNlAM)  that  offers  better 
features.  ENIAM  appears  to  be  a  method  worthy  of  ample  consideration  in  vast  C^I  systems, 
equipped  with  intelligent  modules. 


4.S  Knowledgebase  speciHcation 

4.5.1  Specification  requirements 

An  important  activity  during  expert  system  development  is  the  definition  of  a  conceptual  model. 
The  coiKeptual  model  must  be  a  consistent  representation  of  the  knowledge  domain  in  which,  just 
like  in  database  applications,  a  distinction  is  made  between  a  knowledge  scheme  (the  definitions 
of  all  existing  facts  and  relation^ps)  and  the  actual  knowledgebase.  The  advantage  is  that  now 
the  knowledgebase  can  be  discussed  in  abstraa  terms.  Consistency  and  completeness  can  be 
controUed  by  means  of  constraints  that  are  imposed  upon  the  actual  knowledgebase.  A  conceptual 
model  must  be  made  using  a  method  that  explicitly  defines  the  distinction  between  facts  and  their 
definition.  The  previous  paragraph  labels  NIAM  as  a  prominoit  candidate  in  this  respect. 
Meanwhile,  colleagues  of  Nijssen,  the  devekqxr  of  NIAM,  have  adtted  extensions  to  this  method 
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[Creasy89].  The  core  of  these  is  formed  by  the  possibility  to  formally  represent  constraints  (e.g. 
derivation  and  inference  rules)  in  what  is  called  E(xtended)NIAM.  This  is  the  first  step  in  the 
direction  of  a  more  extensive  integrity  control  within  this  method  because  the  representation  of 
constraints  makes  it  'simple'  to  generate  executable  Prolog  programs.  These  programs  (being  the 
exemplification  of  the  constraints)  can  be,  with  the  help  of  algorithms  that  will  be  dealt  with  later 
in  this  study,  tested  for  consistetKy  and  completeness.  This  aspect  has  already  been  the  subject  of 
investigation  by  [Creasy88]. 

Based  on  [Wintraecken8S]  the  specification  must  satisfy  certain  requirements.  It  must  be  in 
accordance  with  the: 

•  conceptual  principle; 

•  \00%  principle; 

•  natural  language  principle. 

Conceptual  principle 

A  conceptual  grammar  is  a  description  of  all  rules  that  prescribe  what  situations  and  transitions 
between  situations  may  occur  and  what  the  meaning  is  of  the  data  that  are  stored  in  the 
knowledgebase.  The  grammar  consists  of  a  specification  of: 

•  elementary  sentence  types  to  which  elementary  deep  structure  sentences  must  belong,  that 
describe  the  contents  of  the  knowledgebase;  this  implies  the  use  of  the  notions  object, 
relationship  and  constraint; 

•  constraints. 

The  grammar  describes  only  the  conceptual  aspects  of  the  information  exchange  with  the  expert 
system,  i.e.: 

•  describe  only  the  meaning  of  the  data; 

•  no  realization  aspects,  i.e.  im(Hementation'independeitt; 

•  no  description  of  the  internal  /  external  form  of  representation; 

•  indicates  what  exclusively  must  be  described  by  the  grammar. 

100%  principle 

The  grammar  describes  all  conceptual  aspects  of  the  knowledge  exchange  with  the  expert  system; 

•  the  nreaning  of  all  data  is  described  in  the  grammar, 

•  no  other  data  are  exchanged  with  the  expert  system  other  than  those  described; 
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•  when  interpreting  data,  users  may  only  use  rules  from  the  grammar, 

•  never  use  the  knowledge  about  reality,  implicitly  present  in  the  users;  describe  everything 
explicitly. 

Natural  Language  Principle 

Knowledge  can  always  be  described  by  means  of  elementary  deep  structure  sentences  of  a  natural 
language: 

•  the  grammar  is  described  by  means  of  a  natural  language; 

•  all  implicit  knowledge  is  made  explicit. 

The  conceptual  principle  takes  care  of  the  consistency  and  completeness  of  specifications.  The 
completeness  is  guaranteed  by  the  100%  principle.  The  use  of  a  formal  language  for  defining  the 
specifications  leads  to  conciseness.  Nevertheless,  the  readability  is  ’good’  because  the  principle  of 
natural  language  is  used.  Portability  implies  independence  from  implementation-dependent 
matters.  Here  too,  the  conceptual  principle  is  important.  The  adaptability  remains  to  be  discussed. 
The  conceptual  principle  and  the  100%  principle  are  conditional  for  this. 


4.5.2  Constraints 

Vital  to  NIAM  (and  ENIAM)  is  the  principle  of  describing  reality  with  the  help  of  a  natural 
language.  In  the  development  of  expert  systems,  use  is  made  of  knowledge/information  that  has 
been  recorded  in  writing  (manuals  etc.)  on  the  one  harxl,  and  on  the  other  hand,  knowledge  that  is 
based  more  on  experierKe  that  has  been  derived  from  an  expert  through  interviews.  The  intention 
of  knowledge  acquisition  is  to  add  a  structure  to  this  jumble;  a  conceptual  model  of  the 
knowledge  domain  must  be  designed. 

Reality  consists  of  a  set  of  objects  that  are  relevant  within  a  certain  knowledge  domain.  This 
thesis,  based  on  [WintraeckenSS],  implies  that  information  or  communicable  knowledge  is 
nothing  but  making  a  statement  or  an  assertion  about  objects  vnthin  this  reality.  A  fact  is  the 
elementary  assertion  that  establishes  a  relationship  between  objects  in  this  reality.  A  feature  of  an 
elementary  assertion  is  that  it  only  contains  a  single  faa.  Every  non-elementary  assertion  can  be 
bixAen  dowm  into  elementary  assertions. 
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The  objects  in  an  elementary  assertion  can  be  divided  into  lexical  and  non-lexical  ones.  Lexical 
objects  can  somehow  be  represented  in  the  form  of  letters,  niunbers  and/or  other  symbols  and 
may  be  seen  as  names  or  references  to  other  objects.  Non-lexical  objects  carmot  be  represented. 
The  elementary  assertion:  'Jones  is  the  family  name  of  an  employee'  contains  the  name  'Jones'  as  a 
lexical  object  and  'employee'  as  a  non-lexical  object  which  is  referenced  by  means  of  the  name. 

NIAM  substantiates  reality  with  the  help  of  binary  facts,  i.e.  facts  in  which  two  objects  from 
reality  always  play  a  role.  A  distinction  is  made  between  two  types  of  binary  facts:  bridges  and 
ideas.  A  bridge  is  a  fact  that  relates  to  a  lexical  and  a  non-lexical  object;  it  forms,  as  it  were,  a 
bridge  between  two  worlds.  An  idea  is  a  fact  that  relates  to  two  twn-lexical  objects;  the  name  idea 
was  chosen  because  non-lexical  objects  are  discussed  without  mentioning  the  lexical  objects  that 
reference  them.  The  example  in  the  last  paragraph  is  a  bridge. 

One  can  also  speak  of  types:  object  types  Oexical  and  non-lexical)  and  fact  types  (idea  and  bridge 
types).  Introducing  this  abstraction,  it  becomes  possible  to  refer  to  a  set  or  class  of  equivalent 
elements.  The  assertion:  There  are  family  names'  is  an  example  of  the  specification  of  a  lexical 
object  type.  The  assertion  There  are  employees'  specifies  a  non-lexical  objea  type. 

NIAM  is  based  on  the  assumption  that  knowledge  can  be  described  with  facts  and  rules.  Facts  can 
be  seen  as  knowledge  concerning  a  certain  reality  that  may  change  in  time.  Rules  are  elementary 
assertions  that  are  no  facts;  they  define  which  facts  can  be  part  of  communicable  knowledge  and 
which  cannot.  All  facts  are  incorporated  in  the  database,  whereas  the  grammar  contains  a 
description  of  all  the  rules. 

An  important  type  of  rules  are  the  constraints.  These  are  rules  that  are  no  specification  of  object 
types  or  fact  types;  they  restrict  the  facts  that  are  allowed  on  the  basis  of  the  definition  of  object 
types  and  faa  types.  Constraints,  in  their  turn,  can  be  subdivided  in  static  rules  and  transition 
rules.  Static  rules  define  which  facts  may  be  present  in  the  database  at  any  given  time;  they 
describe  states.  Transition  rules  define  which  changes  are  allowed  in  the  database;  they  describe 
transitions  from  one  state  to  another. 

In  general,  the  information/knowledge  analysis  with  the  help  of  NIAM  is  conducted  along  the 
following  lines.  Reality  must  first  be  described  in  words.  Then,  this  description  must  be  unwound 
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into  elementary  assertions.  Subsequently,  these  elementary  assertions  are  divided  in  facts  and 
rules,  and  are  described  formally. 

NIAM  does  not  make  a  distinction  between  graphic  and  non-graphic  constraints.  The  first  can  be 
represented  in  information  structure  diagrams  (ISD)  using  certain  symbols.  The  latter  must  be 
added  to  an  ISD  by  means  of  text. 

A  graphic  constraint,  for  instance,  is  the  unicity  rule  that  indicates  whether  there  is  a  1:1,  1:M  or 
N:M  relationship  between  objects.  Non-grafrfiic  constraints  can  be  divided  into  value  rules, 
cardinality  rules,  rules  determining  subtypes,  derivation  rules,  transition  rules  and  inference  rules. 


4.5.3  An  extension  of  NIAM:  E(xtended)NIAM 

An  important  category  of  constraints  is  the  non-graphic  category.  They  share  the  property  that 
they  cannot  be  graphically  defined  by  the  information  structure  diagrams  GSD)  of  conventional 
NIAM.  Normally,  they  are  added  to  the  graphic  representation  in  natural  language.  This  is  in 
some  respects,  a  disadvantage  since  an  ISD  is  more  inviting  to  the  user  (domain  expert)  during 
validation  of  the  conceptual  model  than  is  a  piece  of  text.  In  the  application  of  NIAM  during  the 
development  of  a  knowledgebase,  it  is  necessary  to  use  non-graphic  constraints.  Especially 
inference  rules  are  quite  similar  to  what  is  called  production  rtiles  in  other  methods.  Referring  to 
the  KADS  method  [Breuker87],  conventional  NIAM  can  offer  a  better  support  at  domain  level 
than  the  usual  AI  development  methods.  Inference  and  domain  level  can  only  be  partly  covered 
by  non-graphic  constraints. 

The  advantages  of  the  NIAM  method  are  the  clear  distinction  made  between  type  and  instances, 
and  the  fact  that  (a  number  of)  constraints  can  be  defined  graphically.  Without  compromising 
these  points,  extended  NIAM  must  be  able  to  add  more  semantics  to  the  information  structure 
diagrams.  [Creasy89]  has  invented  an  adapted  method  called  E(xtended)  NIAM.  This  method  not 
only  offers  the  possibility  (as  does  NIAM)  to  define  'set-oriented'  constraints  graidiically,  but  also 
'member-oriented'  ones.  Examples  of  constraints  based  on  sets  are  unicity,  exception,  and  totality 
rules.  A  member-oriented  constraint  makes  a  statement  about  the  occurrence  or  norKxxutrence  of 
a  member  within  a  certain  set 


TNO  report 


P;igc 

43 


The  extension  is  based  upon  existential  gra{^  [Sowa84],  a  graphic  representation  of  logical 
statements.  A  small  number  of  symbols  is  used  that  has  the  same  ex[xessive  power  as  first  order 
logic.  The  existential  graphs  consist  of  two  parts,  alpha  and  beta.  The  alpha  part  contains  three 
basic  elements,  a  sheet  of  assertion,  a  graph  and  a  cut  The  sheet  of  assertion  may  be  seen  as  a 
model  of  reality.  An  instance  of  a  gra{di  is  equivalent  to  a  prt^sition  (a  statement  about  reality). 
Gre^  instances  are  shown  on  the  sheet  of  assertion  and  are  considered  to  be  true.  The  cut  is  a 
negation  of  a  graph  and  is  represented  as  a  line  drawn  around  the  graph.  Examples  of  the  alpha 
part  of  existential  graphs  are  shown  in  the  next  figure. 


P 


q 


equivalent  with  p  a  q 


P 


equivalent  with  pA-iq 


equivalent  with  -i  (p  a  q),  or  p  -»  q 


Figure  4.1:  Examples  of  existential  grr^hs  (alpha  part). 


It  appears  that  existential  graphs  implicitly  contain  only  a  few  logical  operators,  i.e.  conjunction 
artd  negatioa  There  is,  however,  another  element  that  steins  fnnn  first  order  logic,  namely  the 
existential  quantification  3.  By  means  of  a  line  of  identity,  individual  objects  in  reality  are 
depicted.  The  beta  part  is  based  on  predicate  logic  approadi,  whereas  the  alpha  part  is  based  on 
prxqwsition  logic.  Examples  can  be  found  in  Figure  4.2. 
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P 


equivalent  with  3X  p(X) 


p - q 


equivalent  with  3X  (p(X)  a  q(X)) 


equivalent  with  3X  -i  p(X) 


equivalent  with  -<  (3X  p{X)),  or  VX  pPC) 


equivalent  with  3X  3Y  (p(X)  a  q(Y)  a  X  st  Y) 


Figure  4.2: 


Examples  of  existential  graphs  (beta  pan). 
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4.S.4  Specification  in  E(xtended)NIAM 

From  the  previous  paragraph  it  appears  that  it  is  relatively  easy  to  define  an  inference  rule  (IF 
condition  THEN  action)  with  the  help  of  ENIAM.  In  this  paragraph  it  will  be  demonstrated,  using 
a  small  example,  that  (recursive)  constraints  can  be  represented  with  ENIAM  ISDs. 

This  may  be  illustrated  tvith  the  help  of  the  ancestor  ixoblem.  Solving  this  involves  as  a  first  step, 
determining  when  person  A  qualifies  as  an  ancestor  of  person  B.  There  are  rwo  distinct  cases: 

1.  A  is  an  ancestor  of  B  if  A  is  a  parent  of  B; 

2.  A  is  an  ancestor  of  B  if  A  is  a  parent  of  X  and  X  is  a  parent  of  B. 

If  one  has  at  one's  disposal  a  set  of  facts  defining  who  is  the  parent  of  which  person,  then  one  can 
detennine  the  set  of  ancestors  of  any  person  by  applying  a  recursive  rule.  Defining  a  derivation 
rule  as  is  the  case  with  case  2,  will  not  be  a  problem  with  ENIAM  because  it  is  possible  to  discuss 
the  individual  elements  of  a  set.  The  ISD  is  shown  in  figure  4.3. 


parent 


ancestor 


Figure  4.3: 


ENIAM  information  structure  diagram  for  the  ancestor  problem. 
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The  arrow  over  fact  type  Parent  and  the  asterisk  over  Ancestor  have  die  same  meaning  as  in 
NIAM,  a  M:N  relationship  and  a  derivaUe  fact  type,  respectively.  According  to  NIAM 
conventions,  the  ISD  should  exclusively  consist  of  the  object  type  Person  and  the  fact  types 
Parent  and  Ancestor.  The  two  rules  mentioned  before  that  give  a  definition  of  the  notion  ancestor, 
could,  in  NIAM,  only  have  been  added  to  the  ISD  as  a  small  piece  of  text  ENIAM  allows  graphic 
representation  with  the  extra  advantage  of  each  and  every  constraint  being  defined  within  one  and 
the  same  formalism.  The  general  structure  of  an  if-then  rule  is  depicted  in  the  enlargements  on  the 
left  and  right  side  of  the  original  ISD.  The  existential  graph  on  the  left  contains  both  fact  types 
Parent  and  Ancestor  as  in  case  1.  Referring  to  the  last  example  of  figure  4.2,  p  stands  for  the  fact 
type  Parent  and  q  for  the  fact  type  Ancestor.  There  are  two  lines  of  identity  because  binary  fact 
types  are  involved  here.  Each  of  these  denote  a  single  member  of  a  certain  set.  The  left  pan  of  the 
ISD  in  figure  4.3  is  equivalent  to; 

XfPersonA,  PersonB: 

( parent(PersonA, PersonB)  anccstor(PersonA,PersonB) ) 

The  fact  type  Parent  occurs  mice  on  the  right  side  whereas  Ancestor  occurs  twice,  according  to 
the  second  (indirect)  case.  This  is  equivalent  to; 

VPersonA,  Intermediary,  PersonB: 

( parenl(PersonA,Intermediary)  a  ancesioiflntermediary  J»ersonB) 

-♦  anccstorCPersonA  J^rsonB) ) 

When  these  two,  logical  expressions  are  joined  together  with  the  help  of  a  logical  or-operator,  the 
foOowing  expression  results,  completely  defining  the  ISD  in  frgure  4.3: 

VPersonA,Intermediary  J^rsonB : 

( parent(PersonA,PersonB)  v 

patent(PersonA,Inteimediary)  a  ancestor  (Intermediary J^rsonB) 

-» ancestor(PersonAJ^rsonB)). 

The  logical  expressions  given  in  this  paragraph  can  be  sim|dy  transformed  into  an  executable 
formalism.  i.e.  the  Prolog  programming  language.  Each  of  the  two  cases  in  whidi  the  ancntor 
problem  can  be  broken  down,  lead  to  a  line  of  Prolog.  The  or-ielation^p  that  exists  between 
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these  lines  is  taken  care  of  by  the  interpretation  mechanism  of  Prolog;  if  the  first  line,  after 
matching  the  arguments,  does  not  lead  to  success,  the  second  will  be  tried.  During  the  direct 
transformation  of  an  ENIAM  specification  into  Prolog,  fact  types  are  expressed  by  binary 
predicates.  The  logical  expressions  given  before  only  have  to  be  duplicated  in  reverse.  This  is 
shown  in  figure  4.4. 


ancestor (PersonA, Persons) 

parent (PersonA, PersonB) . 

ancestor (PersonA, Persons) 

parent (PersonA, Intermediary) , 
ancestor (Intermediary, Persons) . 

Figure  4.4:  Ancestor  problem  in  Prolog. 

First,  an  executable  version  of  Prolog  is  made,  to  be  able  to  check  the  integrity  of  the  conceptual 
model  of  the  ancestor  problem.  The  consisteiKy  and  completeness  of  this  program,  aixl  indirectly, 
the  conceptual  model,  can  now  be  investigated  using  the  algorithms  of  chapter  6. 
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5  THE  EXPERT  SYSTEM  AS  A  PRODUCT 

5.1  Intnxluction 

Because  the  subject  of  produa  quality  comprises  too  many  aspects  to  fully  investigate  within  the 
scope  of  this  project,  a  selection  has  been  made  from  various  quality  criteria.  The  selection 
consists  of  integrity,  testability  and  maintainability,  given  concrete  form  in  constraints  for 
consistency,  completeness  and  other  forms  of  integrity  as  well  as  modularity,  respectively. 
Integrity  constraints  may  be  seen  as  a  counterbalance  for  the  poor  testability  and  can  contribute  to 
proper  maintainability.  Modularity  plays  a  role  in  testability  as  well  as  maintainability. 

The  selection  of  the  issues  mentioned  above  does  not  mean  of  course  that  the  others  are  not 
important.  They  arc,  however,  somewhat  more  difficult  to  handle  without  coupling  them  to 
specific  domains  (e.g.  computational  efficietKy)  or  perfonnance  specificaticms  (e.g.  fail-safe  for 
real-time  systems),  or  they  arc  less  important  to  the  quality  problem  in  expert  systems.  Portability, 
for  instance,  is  an  aspect  that  is  not  or  just  barely  concretized  in  expert  systems  but  in  software. 
User  friendliness  (quality  of  the  user  interface)  is  an  aspect  that  by  many  is  considered  to  be  most 
important.  Matters  like  hypertext  and  user  models  can  greatly  enhance  the  quality  of  these 
(Conklin87,  Schneidcr84].  It  can,  however,  hardly  be  seen  as  expert  system  specific,  apart  from 
the  intrinsic  quality  of  the  explanation  that  must  be  generated  by  the  system.  As  far  as  domain- 
independent  guidelines  can  be  given  for  this,  these  result  in  an  explicit  representation  of  meta¬ 
knowledge.  which  can  be  seen  as  a  form  of  structuring  or  modularity  (Wognum89].  Modularity  is 
a  a  characieristic  of  software  that  is  very  important  while  studying  expert  systems.  This  study 
looks  at  it  from  the  angle  of  testability,  because  integrity  results  from  it. 


5.2  Testalnlity  of  expert  systems 

5.2. 1  Conventional  testing  methodologies 

Methods  to  guarantee  the  reliability  of  software  can  be  subdivided  into  methods  that  'embed' 
(enhanced)  reliability  (e.g.  structured  analysis,  design  and  implementation  methods)  and  methods 
that  test  reliability  afterwards.  The  overlapping  notions  testing,  evaluating,  validating  and 
verifying  belong  to  this  last  group.  Over  the  years,  an  extensive  range  of  methods,  paradigms  and 
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hints  has  been  developed  for  testing  software.  Starting  principle  of  this  paragraph  is  a  testing 
method  as  described  in  [Myers79].  Whereas  this  method  is  very  useful  for  testing  conventional 
programs  like  interpreters  and  operating  systems,  it  seems  to  be  inadequate  for  expert  systems  and 
(to  a  somewhat  lesser  extent)  other  data-dominated  systems  such  as  databases  [£>eutsch82].  It 
remains  a  question  by  what  it  is  caused.  To  get  a  better  insight  into  this  issue,  we  will  start  to 
review  some  techniques  of  Myers'  testing  method.  Testing,  the  Myers-way,  must  partly  be  done 
by  executing  the  software  and  partly  by  using  one's  brains. 

To  the  mental  techniques  belong,  among  others,  inspection  and  walkthrough.  Walkthrough  comes 
down  to  investigate  how  the  interpreted  program  will  react  to  certain  test  cases  using  the  source 
code.  As  in  every  form  of  testing,  the  expeaed  or  desired  output  must  be  specified  before  actual 
testing.  Inspection  assumes  two  different  forms:  a  verbal  paraphrase  of  the  programmer  of  what 
the  program  does  (using  the  source  code)  and  nmning  down  a  check-list  with  likely  errors.  The 
check-list  given  in  [Myers79]  contains,  for  instance,  items  such  as  'index  within  bounds?'.  This  is 
of  little  use  when  one  programs  in  a  language  like  PASCAL;  if  a  reasonably  well-designed 
compiler  is  used,  a  possible  'index  out  of  bounds'  error  will  be  detected  automatically  during 
compilation. 

Testing  through  execution  also  consists  of  two  types,  i.e.  black  box  and  white  box  testing.  The 
first  type  considers  the  system  or  module  to  be  a  black  box:  otdy  the  result  (the  ouqnit)  is 
important  when  feeding  certain  input  How  this  software  establishes  its  output  is  of  no  importance 
to  black  box  testing.  Because  it  is  virtually  impossible  for  most  software  to  try  all  possible  input 
values,  it  is  attempted  to  compile  a  set  of  test  cases  that  is  as  efficient  as  possible.  Efficient  here 
means;  a  small  number  with  a  large  probability  of  errors  manifesting  themselves.  Experience 
teaches  us  that  the  value  range  of  input  can  be  divided  into  so-called  equivalence  classes, 
characterized  by  the  fact  that  two  test  cases  from  die  same  equivaloice  class  will  give  rise  to  die 
same  errors(if  resulting  in  an  error  at  all).  Of  course,  the  detennination  of  equivaloice  classes  is  a 
gamble  in  case  of  a  black  box:  only  the  specification  gives  a  lead.  Yet,  we  can  make  an  educated 
guess  on  the  basis  of  that  To  select  ,  for  examine,  equivalence  classes  for  a  module  diat, 
according  to  a  (concise)  specification,  'extracts  the  root  of  a  number'  one  could  reason  as  follows. 
One  can  distinguish  the  subsets  xl  Vx  not  deflned  as  xl  Vx  >>  x  en  xl  Vx  <  x  as  far  as  the 
function  that  x  adds  to  the  root  of  x  within  die  real  numbers.  Besides  diat,  mie  can  make  a 
distinction  between  integers  and  fractions  (the  distinction  between  rational  and  real  is  not 
impemant  of  course:  normally,  only  rational  numbers  can  be  represented).  Besides  the  notion  of 
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equivalence  class,  the  notion  of  limit  plays  a  role:  it  is  good,  heuristic  practice  to  use  test  cases 
with  values  on  the  very  limit  of  their  equivaleitce  class.  In  the  case  on  hand,  this  could  lead,  for 
example,  to  the  set  of  test  cases  (-1,  ?),  (0,  0).  (0.01,  0.1),  (1,  1),  (2.25,  1.5),  (1.0E30,  1.0E15). 
Because  of  the  heuristic  character  of  the  use  of  equivalence  classes  and  limits  it  cannot  be 
determined  objectively,  what  exactly  is  the  'best'  set  of  test  cases.  Furthermore,  an  optimum  state 
is  always  a  balance  between  enhancing  the  error-finding  probability  and  limiting  the  time 
involved  in  testing. 

Contrary  to  black  box  testing,  white  box  testing  uses  the  structure  of  tlK  source  code.  The  thought 
behind  the  white  box  techniques  presented  by  Myers,  is  that  the  ideal  set  of  test  cases  will  literally 
cover  every  nook  and  cranny  of  a  program.  This  ideal  can  best  be  approached  using  path 
coverage.  In  path  coverage,  every  possible  path  of  execution  in  the  program  is  covered  by  (at 
least)  one  test  case.  In  general,  the  demand  for  a  path  coverage  leads  to  unacceptable  costs  (it 
must  be  kept  in  mind  that  a  path  that  runs  through  one  or  other  repetitive  loop  differs  from  the 
path  that  runs  through  one  and  the  same  loop  for  three  times).  Therefore,  one  has  established  a  set 
of  coverage  criteria,  that  will  be  discussed  in  order  of  increasing  importance. 

a.  Statement  coverage:  Every  statement  must  be  executed  during  testing  at  least  otKe. 

Example:  To  test  'IF  cl  OR  c2  THEN  a1  ELSE  a2'  with  a  statement  coverage,  you  need  at  least 
two  test  cases:  one  executing  al  and  a  second  executing  a2. 

b.  Decision  coverage:  Every  branch  (the  result  of  a  decision)  must  be  run  through  during 
testing  at  least  once. 

Example:  To  test  'IF  cl  OR  c2  THEN  al'  with  a  decision  coverage,  you  need  at  least  two  test 
cases:  one  resulting  a  TRUE'  for  'cl  OR  c2'  (and  thus  executing  al)  and  one  resulting  in  FALSE. 
Note  that  one  test  case  would  suffice  as  a  statement  coverage.  Decision  coverage  imfdies 
statement  coverage. 

c.  Condition  coverage:  Every  condition  (term  in  a  logical  expression)  must  at  least  produce 
one  instance  of  TRUE  and  me  of  FALSE  during  testing. 


Example;  To  test  'IF  cl  OR  cl  THEN  al  ELSE  a2'  with  condition  coverage  you  need  at  least  two 
test  cases;  one  for  instance,  with  cl  resulting  in  TRUE  and  cl  in  FALSE  and  one  producing  the 
reverse.  Note  that  in  this  case  the  requirements  for  decision  coverage  or  even  statement  coverage 
are  not  satisfied. 

d.  Decision  /  Condition  coverage:  Both  decision  coverage  as  well  as  condition  coverage 
must  be  satisfied. 

Example:  To  satisfy  the  aforementioned  example  two  test  cases  will  satisfy,  i.e.  one  with  cl  and 
cl  both  qualify  as  TRUE  and  one  where  both  yield  FALSE.  Decision  /  condition  coverage  is  a  far 
better  criterion  than  condition  coverage:  the  test  set  covers  more  of  the  program,  but  need  tK>t  be 
larger! 

e.  Multiple  condition  coverage:  During  testing,  the  tuple  of  conditions  in  a  decision  must  be 
run  through  every  possible  value  at  least  once. 

Example:  To  test  'IF  cl  OR  cl  THEN  al  ELSE  a2'  with  multiple  coverage  you  need  at  least  2  to 
power  of  2  =  4  test  cases,  one  with  both  conditions  resulting  in  TRUE,  one  making  both  FALSE 
and  two  with  one  resulting  in  FALSE  and  the  other  TRUE.  Multiple  condition  coverage  im^dies 
decision  /  condition  coverage  but,  in  general,  requires  more  (and  sometimes  considerably  more) 
effort  in  testing.  Multiple  condition  coverage  does  not  imply  path  coverage  in  any  respea,  not 
even  in  a  program  without  repetitive  loops  (a  decision  tree).  All  possible  combinations  of 
condition  results  would  be  tested  for  each,  individual  decision.  Path  coverage  would  imply  that  all 
possitde  combinations  of  decision  results  (per  program)  would  be  tested. 

It  is  reasonable  to  assume  that  ix>n-nested  decisions  can  be  examined  indq^endently  horn  each 
other.  This  is  not  true,  however,  in  the  case  of  nested  decisions,  especially  when  muldide 
condition  coverage  is  required.  (Myer579]  does  not  give  a  clear  stat^ent  on  this.  Nevertheless,  it 
is  repeatedly  indicated  that  multii^e  condition  coverage  is  a  ttecessary  prerequisite  for  prop^ 
testing.  When  you  consider  that  nested  decisions  may  be  seen  as  a  single  decision,  a  maximum 
number  of  conditions  of  about  20  would  tx>t  seem  impossiUe  for  extensive  applications.  This 
would  come  down  to  a  magnitude  of  abmit  a  million  test  cases.  It  is  therefore  safe  to  assume  that 
cmKhtitMts  in  nested  decisions  do  rxK  influence  each  other  or  at  least  not  always. 
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5.2.2  Af^licability  of  conventional  testing  methodologies  on  expert  systems 

Considering  the  statements  in  various  publications  about  the  intricate  flow  of  control,  impairing 
proper  testing  of  expert  systems,  and  the  'complications  during  the  testing  of  data-dominated 
systems',  it  is  concluded  that  the  specific  problems  inherent  to  testing  an  expert  system  belong  to 
the  declarative  part  (the  knowledgebase(s))  and  not  to  the  inference  ragine  (shell)  [Llinas87 
Myers79,  Deutsch82].  It  is  held,  in  general,  that  the  advantage  of  explicit  (declarative)  knowledge 
representation  over  implicit  (procedural)  lies  in  the  fact  that  shell  and  knowledgebase  may  be 
tested  separately.  Ehiring  the  discussion  of  the  applicability  of  the  aforementioned  white  and  black 
box  testing  methods,  it  is  assumed  that  the  shell  will  contain  no  bugs  (anymore). 

Equivalence  class,  the  leading  concept  in  black  box  testing  can  (mly  be  useful  in  a  specification 
that  is  either  very  detailed  or  will  be  approached  identically  by  different  persons.  In  Mack  box 
testing  it  is  also  desirable  that  the  number  of  test  cases  necessary  is  not  too  large.  [Doyle85] 
remarks  that  'in  the  AI  approach  (...)  the  interpreted  specification  functions  as  an  inexpensive,  but 
complete  prototype'.  Here,  the  knowledgebase  is  considered  to  be  a  (sub)specification  that 
together  with  the  (conventional)  specification  of  the  shell,  forms  a  specification  of  the  expert 
system.  This  may  be  true  when  the  knowledgebase  is  relatively  small  and  orderly,  but  should  this 
not  be  the  case  then  it  seems  inappropriate  to  designate  the  knowledgebase  as  a  specification.  A 
requirement  that  must  part  of  every  specification  is  that  its  validity  can  be  checked  without 
problems.  Should  the  knowledgebase  be  poorly  organized,  however,  then  this  will  not  be  possible. 
The  main  question  now  is  whether  in  such  a  case  a  specification  of  the  knowledgebase  can  be 
formulated  that  can  alleviate  the  testing  and  maintainability  problems  observed,  or  eliminate  them 
altogether.  After  having  read  the  previous  chapter,  the  reader  will  not  be  amazed  that  our  answer 
to  this  question  ranges  from  a  cautious  (as  far  as  elimination  is  coiKemed)  to  an  unconditional  (as 
far  as  alleviation  is  concerned)  yes.  Doyle's  statement  cited  previously  seems  to  hold  true  for  the 
rule  part  of  an  expert  system.  In  sudi  a  situation  Uadt  and  white  box  methods  coindde.  The  tmly 
way  then,  to  enhaiKC  the  orderliness  of  product  alias  spedficatitm  is  to  reformulate  die  rules, 
using,  for  instance,  other  domain  ctmcepts  or  a  totally  different  approach  producing  more 
modularity.  It  goes  without  mention  that  generally  speaking,  this  wiU  not  be  a  amide  task. 

The  use  of  a  limit  in  test  cases  will  in  general  be  less  fhiitful  than  in  eiqiert  system,  simidy 
because  the  variatdes  are  symbolic  more  often  and  thus  discrete  or  even  binary.  Illegal  values, 
however,  can  iday  a  very  useful  role,  espedally  there  v4ieie  making  test  cases  gets  complicated 
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because  the  maker  is  not  able  sui^Iy  the  desired  (correct)  ouQHit  to  the  ii^t.  Take,  for  instance, 
an  influence  graph  of  physiological  variables  with  observable  [Aienomena  at  the  extremes 
(symptoms  or  syndromes).  Usually,  one  has  to  be  an  expert  (a  j^iysician)  to  come  up  with  a 
(minimal)  commcm  cause  of  the  symptoms  of  hypertension  (hi^  blood  pressure)  and  haematuria 
(blood  in  urine).  A  grain  of  common  sense  suffices,  however,  to  infer  that  a  combination  of  coma 
and  hyperventilation  is  illegal.  The  test  case  (coma,  hyperventilation,??)  can  thus  be  made  without 
the  help  of  an  expert. 

(This  test  case  is  derived  from  daily  practice;  it  immediately  demonstrated  an  essential  design 
error  in  the  respective  algorithm).  When  an  expert  system  is  subjected  to  multiple  condition 
coverage  (about  nested  conditions),  then  this  will  soon  lead  to  an  unacceptably  high  number  of 
test  cases.  The  fact  that  an  expert  will  usually  have  to  assist  in  connecting  the  desired  output  to 
the  input  of  the  test  cases  makes  it  even  worse.  After  all,  the  availability  of  experts  is  a  familiar 
bottle-neck  in  the  development  of  expert  systems. 

Holding  on  to  limited  multiple  condition  coverage  (i.e.  within  a  decision)  is,  however,  dubious  in 
many  cases,  especially  in  the  case  of  a  decision  tree.  A  subdivision  (preferably  hierarchically)  of 
the  knowledgebase  in  small  modules  is  therefore  desired.  Multiple  condition  coverage  can  then  be 
demanded  in  every  individual  module.  If  these  modules  are  not  too  large  and  legitimately  claim 
the  title  of  module,  in  other  words,  form  conceptual  units  without  too  much  interaction,  then  such 
an  approach  is  undoubtedly  optimal.  Which  demonstrates  the  vital  importance  of  structuring  and 
modularity  to  testability.  Needless  to  say  perhaps,  is  the  remark  that  the  aforementioned  list  of 
requirements  has  been  written  down  without  taking  into  account  its  feasibility.  It  is  merely  a 
sketch  of  the  structure  that  would  be  ideal  for  testing  and  maintenance.  In  practice  things  would 
be  too  rigid  (in  hierarchical  structuring,  for  instance)  to  serve  as  an  effective  guideline. 


S.3  Structure  and  modularity 

S.3. 1  Modularity  in  conventiotud  software 

In  a  traditional  sense  a  software  design  is  structured  properly,  when  the  modules  possess  a  certain 
independence  from  each  odier  artd  every  module  is  internally  odierent  from  a  functional  point  of 
view,  i.e.  actions  within  die  module  are  reitted  to  each  other.  The  first  aspect  is  called  cou(ding, 
the  second  is  called  cohesion.  The  ctmcepu  ate  not  mutually  independettt.  As  modules  obtidn  a 
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Stronger  cohesion,  generally  a  weaker  coupling  will  suffice  to  let  the  whole  system  function 
properly. 

Coupling  is  the  degree  of  dependence  between  two  modules.  This  rather  abstract  definition  can  be 
operationalized  as  the  (xobability  that  during  encoding,  debugging  or  modification  of  a  module, 
the  contents  of  another  module  must  be  taken  into  consideraticm.  The  coupling  between  modules 
comprises  the  following  four  aspects: 

a.  Nature  of  the  coupling  between  modules 

A  coupling  through  sending  messages  atal  their  reactions  as  is  used  in  objea-oriented 
programming,  is  of  an  entirely  different  nature  than  the  mutual  use  of  a  global  data  area 
(common  envirorunent  coupling).  The  coupling  through  import  and  export  of  variables 
and  subroutines  lies  more  or  less  in  between. 

b.  Complexity  of  irformadon  exchange 

This  must  be  measured  in  terms  of  the  number  of  items  that  is  exchanged  between 
modules;  not  the  amount  of  data  exchanged  expressed  in  bytes. 

c.  Time  of  instantiation  <fa  connection  between  modules 

Conneaiotts  instantiated  at  compile  time  are  preferred  to  those  instantiated  at  run-time. 

d.  Nature  of  the  information  exchanged 

On  the  basis  of  this,  [Gane79]  distinguishes  three  types  of  coupling,  i.e.  data-coupling, 
control-coupling  arid  content-coupling.  The  latter  is  also  called  external  or  pathological 
coupling. 

The  significatKe  that  [Gane79]  gives  to  these  three  terms,  essentiaUy  means  that  data-coufding 
must  be  preferred  to  control-coupling  because  the  first  does  not  influence  the  flow  of  ccmtrol  of 
the  receiving  module  and  the  second  one  does.  The  transfer  of  flags  comes  to  mind  in  the  latter 
case.  Content-coupling  is  inadmissible  altogether.  This  means  that  a  module  covertly  reads  or 
changes  the  value  of  a  variable  in  another  module.  One  can  think  of  a  module  hierarchy,  for 
instance,  that  is  being  skipped  surreptitiously. 
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The  distinction  between  data  and  control-coupling  is  formulated  somewhat  extremely.  It  is 
unavoidable  that  the  flow  of  control  of  the  receiving  module  will  be  influenced.  After  all,  in  every 
subroutine  that  tests  the  value  of  an  argument  (e.g.  in  a  subroutine  that  extracts  toots,  a  test  that 
determines  whether  or  not  the  argument  is  negative)  it  can  be  maintained  that  the  value  of  the 
argument  influences  the  flow  of  control.  This  fonn  of  control-coupling  is  not  all  that  inferior  to 
pure  data-coupling  as  long  as  two  conditions  are  met.  Firstly,  it  must  be  possible  to  specify  the 
influence  of  information  coming  from  the  calling  module  on  the  flow  of  control  of  the  called 
module,  without  referring  to  the  identity  of  the  caller.  Secondly,  the  influence  may  not  have  any 
effect  on  the  flow  of  control  in  the  module  after  the  result  has  been  given  back  to  the  calling 
module. 

As  has  been  remarked  before,  a  surplus  of  coupling  between  modules  can  generally  be  taken  as 
the  result  of  a  lack  of  cohesion  within  modules. 

Usually,  in  software  one  can  discern  six  to  seven  levels  of  cohesion,  i.e.  (in  decreasing  order  of 
preference): 

a.  Functional  cohesion 

Every  element  (subroutine)  in  a  functionally  coherent  module  forms  an  integral  part  of  a 
single  function.  In  other  words,  the  elements  are  a  unit,  procedurally  as  well  as 
conceptually.  As  a  test  for  functional  cohesion,  one  can  use  the  rule  of  thumb  that  it  must 
be  possible  to  express  the  function  of  a  module  in  the  imperative  plus  a  direct  object  (e.g. 
'calculate  the  optimum  solution'  or 'format  the  answer'). 

b.  Sequential  cohesion 

A  sequentially  coherent  module  is  a  cluster  of  a  number  of  functionally  coherent  parts 
where  the  output  of  one  part  serves  as  input  for  the  following  part  (e.g.  'read  line  of  ii^t 
and  delete  spaces'  or  'calculate  solution  and  print  it'). 

c.  Communicative  cohesion 

A  communicatively  coherent  module  consists  of  functionally  coherent  components  that 
have  a  direct  object  in  common  (e.g.  'print  the  solution  and  store  it  in  the  database'). 
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d.  Procedural  cohesion 

Procedurally  coherent  modules  can  emerge  when  the  symbols  of  a  flowchart  are  placed 
into  a  module  (e.g.  process  symbols  in  data  flow  diagrams).  When  this  results  into  a 
conceptual  unit  this  is  called  procedural  ccrfiesion. 

e.  Temporal  cohesion 

The  elements  (subroutines)  in  a  temporally  coherent  module  have  in  common  that  they 
occur  in  the  same  stage  of  execution  (e.g.  initialization). 

f.  Logical  cohesion 

Similar  looking  pieces  of  code  yet  being  different  in  detail  that  are  merged  with  the 
exclusion  of  identical  elements  (except  for  one  item,  of  course)  fonn  a  logically  coherent 
module  (e.g.  a  module  containing  all  input  procedures). 

g.  Coincidal  cohesion 

Coincidally  coherent  modules  consist  of  elements  that  have  no  common  denominator 
other  than  contained  within  one  and  the  same  module. 


5.3.2  Modular  structure  of  knowledgebases 

The  traditional  knowledgebase  consists  of  a  tangle  of  production  rules  without  any  form  of 
modular  structuring.  The  emergence  of  second  generation  expert  systems  has  started  a  first 
incentive  to  structuring:  meta-knowledge  is  separated  from  domain-knowledge.  Gradually,  the 
knowledgebase  programmer  looses  the  freedom  that  was  a  former  pitfall.  The  need  of 
prestructuring  the  knowledge  model  as  a  Reification  of  the  knowledgebase  is  generally 
recognized  nowadays.  There  is,  however,  no  concensus  yet  about  the  form  such  a  prestructuring 
should  have.  The  translation  too,  of  a  structured  specification  into  a  knowledgebase  structure 
usually  lacks  a  clear  descri^Kion. 

Within  the  framework  of  the  KADS  project,  a  three  to  four-layer  knowledge  model  is  used,  i.e.  a 
domain-layer,  a  (cancmical)  inference-layer,  a  task-layer  and  sometimes  (seldmn  in  concrete 
applications)  a  strategy-layer  [FEL1989-148].  The  central  concept  of  KADS  is  that  of  a  genetic 
task.  As  a  good  examine  we  mention  the  heuristic  classification  task  [QanceySS].  KADS. 
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however,  aims  primarily  at  knowledge  acquisition.  It  pays  too  little  attenticm  to  other  relevant 
matters  to  qualify  as  a  development  methodology.  At  first,  the  transition  of  the  knowledge  model 
(as  a  final  produa  of  knowledge  acquisition)  to  software  was  the  poor  cousin.  Recently,  diis  has 
been  improved  considerably  through  fonnalization  of  the  KADS  knowledge  model 
[Akkennans891.  Nevertheless,  issues  on  testing  and  maintenance,  for  instance,  are  not  or  just 
barely  addressed  in  KADS  publications.  G)ntrary  to  [Chandrasekaran87],  where  generic  tasks 
also  play  a  role,  be  it  that  its  content  differ  from  tlK  generic  tasks.  Chandrasekaran  emphasizes,  as 
does  Breuker  c.s.,  the  importance  of  building  bricks  for  knowledge  modelling  that  possess  a 
substantially  higher  level  of  abstraction  than  matters  like  production  rules,  frames  and  networks. 
Building  on  earlier  work  in  the  field  of  dia^osis  and  design,  he  presents  four  generic  tasks  for 
diagnosis,  hierarchic  classification,  hypothesis  evaluation  Cwhat  is  the  concord  between  data  and 
hypothesis?'),  abductive  assembly  (to  join  a  number  of  low-level  hypotheses  together  into  a 
single,  coherent  unit)  and  database  inference  [Chandrasekaran83,84,86].  Besides  these  four, 
Chandrasekaran  mention  two  others  in  relation  to  design:  object-synthesis-through-slection-and- 
refinement-of-plans  and  situational-space-abstraction.  For  aU  six  tasks,  generic  tools  are  available. 
Chandrasekaran  explicitly  mentions  the  undesirability  of  unifonn  knowledge  representation  for 
various,  generic  tasks.  He  sees  intelligence  as  an  interaction  of  fimctional  units  (problem-solvers) 
that  each  use  the  knowledge  representation  and  inference  methods  according  to  their  type. 

The  latter  statement  comes  close  to  the  i^osophy  behind  blackboard  systems  and  object-oriented 
programming.  The  blackboard  approach  was  not  originally  developed  to  improve  quality 
assurance  of  expert  system  development,  but  to  tackle  complex  problems  such  as  speech 
recognition  [Etman80J’ennell77].  As  such,  a  blackboard  architecture  is  eminently  suitable  for 
problem-solving  where  hypotheses  that  are  very  uncertain  at  first  are  accepted  or  rejected  by 
another  (i.e.  by  looking  at  it  from  a  different  angle).  This  approach  to  problem-solving,  however, 
lends  itself  to  many  domains.  Physicians,  for  example,  use  nosological,  epidemological, 
physiological  and  anatomical  knowledge  in  the  reasoning  (e.g.  diagrmsis).  Barristers  use 
jurisprudence  alongside  the  law.  Generalizations  (of  domain-specific  blackboard  systons)  such  as 
AGE/SHELL,  Hearsay-Ill,  BBl,  MXA  and  BLOBS  are  diaracteristic  for  die  ambitions  in  the 
direction  of  general  tqiplicability  of  Uadtboard  technology  [Engelmore88].  Besides  the  use  of 
more  than  one  inference  method  and  a  measure  of  uncertainty  in  many  domains,  there  is  yet 
another  point  duit  makes  the  blackboard  paradigm  an  important  concept  for  expert  systems.  A 
blackboard  architecture  offers  good  possibilities  to  build  in  a  reasonably  advanced  modularity  and 
as  sudi,  deliver  a  more  orderly  product  that  is  easier  to  test  This  aqiect  has,  as  yet,  received  little 
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attention  which  is  probably  due  to  the  fact  that,  until  recently,  the  develofnnent  of  a  lai:ge 
blackboard  system  was  next  to  slavery.  Now  that  more  aiKl  more  blackboard  development 
systems  become  available,  like  BBl  [Hayes-Roth88]  and  GBB  [Coikill88],  Nackboard 
architectures  will  probably  gain  more  acceptance.  Especially  GBB  (Generic  BlackBoard)  is 
interesting  fnxn  a  quality  assurance  point  of  view  because  of  the  strong  emi^iasis  that  is  put  on 
specification  of  the  data  structure  before  imi^ementation  in  the  system.  A  clear  parallel  is  drawn 
with  data  definition  in  database  methodology.  Furthermore,  GBB  pays  much  attention  to 
efficiency  in  entering  of  new  data  and  retrieving  existing  data  (blackboard  objects  in  this  case). 
This  is  an  important  aspect,  given  the  sorrowful  reputation  of  blackboard  systems  in  this  respect. 
Another  generalization  that  particularly  emphasizes  the  usefulitess  of  blackboard  systems  in 
relation  to  quality  assurance  is  the  Edinburgh  Prolog  Blackboard  SheU  [Engelmore88].  As  a 
recent  example  of  blackboard  technology  efforts  of  national  origin,  we  mention  the  legal  expert 
system  Prolexs  (Walker88]. 

Another  trend  is  the  increasing  attention  for  integration  of  database  technology  and  knowledge 
engineering.  This  is  exemplified  by  MEDES,  a  medical  system  that  lies  somewhere  in  between  a 
database  system  and  a  development  environment  for  medical  expert  systems  [DeVriesRobb689]. 
The  domain  expert  uses  software  that  is  more  like  a  database  than  an  expert  system,  since  all  he 
or  she  can  enter  are  relationship  instances.  The  queries  an  end-user  can  pose,  however,  make  the 
system  an  expert  system  to  that  end-user.  MEDES  has  three  layers.  The  inner  layer  is  a  general 
purpose  shell.  Around  that  we  find  a  layer  with  inference  algorithms  and  predicate  definitions  that 
may  (generally)  be  considered  of  importance  to  the  medical  domain.  The  outer  layer  consists  of 
static  domain-  knowledge.  This  layer  (and  in  principle  this  layer  only)  is  filled  by  a  domain- 
expert,  and  he  or  she  can  only  use  those  predicates  that  are  defmed  in  the  middle  layer  (the  so- 
called  subshell).  Because  inference  primitives  to  be  used  in  queries,  ate  defmed  in  the  subshell 
not  accessible  for  the  domain-expert,  it  is  expected  from  such  a  system  that  it  allows  a  npd, 
flexilde  implementation  of  a  new  (medical)  domain  that  is  not  messed  up  by  annoying  typing 
errors.  Of  course,  a  predefined  set  of  predicates  implies  limitations  to  a  domain-expert  entering 
a(nother)  medical  domain.  ExpetieiKe  must  demr)nstrate  whether  this  classifies  as  an  inesolvaUe 
hindnmce  or  a  welcome  limitation.  However,  it  is  certain  that  when  one  oideavours  to  promote 
expert  system  development  form  craft  to  industry,  such  a  limitation  of  liberties  is  an  tq^ropriate 
(and  possibly  the  only)  way. 
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S.3.3  Cohesion  and  coupling  in  knowledgebases 

At  least  as  far  as  the  definitions  in  [Gane86]  and  [Yourdon79]  go,  coiq>ling  and  cohesion  are 
characteristic  of  procedural  software.  This  does  not  mean,  however,  that  these  cannot  relate  to 
knowledgebases:  when  a  knowledgebase  is  subdivided  in  modules  it  is  very  well  possible  that  a 
change  in  one  module  causes  changes  in  other  modules,  e.g.  in  order  to  keep  the  knowledgebase 
consistent  as  a  whole.  In  fact,  [Nguyen87]'s  algorithm  for  consistency  control  uses  a  modular 
subdivision  of  the  knowledgebase  to  perform  a  check  more  efficiently,  be  it  that  the  subdivision 
into  modules  is  done  trfterwards  (i.e.  after  the  development  of  the  system).  Because  this  is  done 
purely  on  the  basis  of  dependency  between  predicates  without  taking  into  account  the  conceptual 
unity  of  modules,  there  is  no  procedural  cohesion  involved.  It  cannot  be  determined  whether  the 
coupling  of  the  modules  in  the  context  tree  is  data  or  control-coupled.  This  depends  on  whether 
during  the  inference  process,  new  (derived)  facts  are  being  stored. 

In  blackboard  architectures  with  a  single  blackboard,  i.e.  not  divided  in  panels,  the  coupling 
between  the  separate  knowledgebases  is  in  principle  a  form  of  common  environment  coupling. 
Usually,  blackboards  are  divided  in  panels  atxl  for  every  .Jiowledgebase  it  is  specified  what 
panels  may  be  read  and  which  may  be  written  on.  In  that  case,  the  coupling  may  be  better 
described  as  object-oriented.  In  both  cases,  this  basically  involves  data-coupling  in  the  sense  that 
a  change  in  one  knowledgebase  produces  no  unwanted  side-effects  in  another  knowledgebase,  at 
least  not  if  the  design  of  the  knowledgebases  has  not  taken  the  functionality  of  those  other 
knowledgebases  into  consideration.  An  exception  to  this  is  changing  the  access  rights  (read  / 
write)  of  a  knowledgebase  on  the  various  blackboard  panels.  Assuming  that  the  access  tights  form 
a  neatly  arranged  unit,  this  will  not  give  any  problems.  Such  a  change  might,  however,  be 
considered  as  a  change  of  the  goal  specification,  something  that  would  not  occur  in  regular 
maintenance.  In  a  Uackboard  system  such  as  Hearsay-II,  the  various  knowledgebases  are  distitKt 
from  each  other,  not  only  procedurally  but  also  conceptually.  Therefore,  rx)  Junctional  cohesion  is 
involved.  Neither  is  module  hierarchy  in  ordinary  blackboard  systems.  Distributed  blackboard 
systems  possess  a  limited  (two-layer)  module  hierarchy  [DurfeeSS]. 
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6  INTEGRITY  CONTROL  ALGORITHMS 

6. 1  Introduction 

Integrity  control  in  databases  has  been  further  developed  than  the  control  of  consistency  and 
completeness  in  expert  systems.  The  so-called  expert  database  systems  or  knowledgebase 
management  systems  lesemUe  as  far  as  implementation  is  concerned,  a  database,  but  as  far  as  the 
outer  texture  is  concerned  (as  well  as  deductive  capabilities),  they  look  more  like  an  expert 
system.  For  some  time  now,  research  has  been  going  on  with  respect  to  (more)  powerful  integrity 
control,  as  far  as  expressional  power  of  the  constraints  and  efficiency  is  concerned. 

Each  of  the  paragraphs  contained  in  this  chapter  comprises  the  description  of  an  algorithm  with 
which  one  can  check  the  integrity  of  a  knowledgebase.  There  are,  however,  important  differences 
between  the  algorithms.  They  can  be  described  as: 

•  deductive  database  perspective  (Decker); 

•  programming  language  perspective  (Shapiro); 

•  production  rule  perspective  (Nguyen). 

6.2  Decker’s  algorithm 

The  algorithm  described  in  [DeckerSd]  is  meant  to  control  the  integrity  of  a  deductive  database.  A 
deductive  database  contains  three  types  of  data.  Beside  the  facts  and  constraints  that  we  also  find 
in  ordinary  databases,  it  also  contains  derivation  rules.  With  these  rules,  other  facts  can  be  derived 
from  those  explicitly  incorporated  (the  extension  of  the  database).  The  derivable  facu  are  only 
implicitly  present  and  are  only  derived  when  triggered  by  a  database  query.  Because  of  the 
presence  of  derivation  rules,  a  deductive  database  qualifies  as  a  knowledgebase.  As  such  it  is  an 
alternative  for  production  rule  systems.  The  integrity  of  a  deductive  database  is  the  integrity  of  dK 
ordinary  database  that  is  obtained  by  the  exhaustive  apfdication  of  derivation  niles.  Whereas  the 
constraints  in  an  ordinary  debase  limit  die  set  of  facts  contained  (explicidy)  in  the  database,  the 
constraints  of  a  deduaive  database  system  relate  to  the  «4ioIe  set  of  (either  explicit  or  derivable) 
facts.  In  daudiases  as  well  as  deductive  databases,  the  term  integrity  check  is  used  whidi  poirtts 
out  that  the  main  concern  is  maintaining  the  integrity  when  changing  the  database  or 
knowtedgdMse,  respectively.  The  algorithms  for  checking  integrity  are  therefore  equipped  to 
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efficienOy  perform  an  integrity  check  after  a  (small)  change  of  the  datafile.  In  the  case  of 
databases  as  well  as  deductive  databases  it  is  assumed  that  the  file  complied  to  the  constraints 
before  the  update  took  place.  With  the  help  of  diis  assumpticm,  constraints  may  be  simplified  in 
terms  of  compile  time.  It  is  now  possible  to  determine  whilst  updating,  on  the  basis  of  contents 
and  type  of  the  update,  which  of  these  simfdified  constraints  must  be  evaluated.  How  this  works 
will  be  exemplified  at  the  end  of  this  paragraph. 

In  deductive  databases,  queries  as  well  as  derivation  rules  are  generally  Horn-clauses.  Hom- 
clauses  are  insufficiently  expressive  to  be  constraints.  Instead,  larger  subclasses  of  first  order 
logic  are  used  such  as  allowed  formulas  [Topor86]  or  range-restricted  formulas 
(Nicolas82,DeckerS6].  Deductive  databases  have  until  now  been  the  subject  of  theoretical 
research,  using  internal  Prolog  databases.  [Decker86]  pays  much  attention  to  working  with  range- 
restricted  formulas,  especially  in  checking  whether  a  first-order  formula  is  range-restricted  and 
the  description  of  range-restricted  formulas  in  the  canonical  form,  the  so-called  range-form.  This 
standard  form  offers  the  possibility  to  write  a  simple  interpreter  in  Prolog  that  can  evaluate  all 
range-restricted  formulas.  Decker's  discussion  about  range-restricted  formulas  is  hard  to  follow 
for  those  that  do  not  have  received  any  training  in  logic  and  logic  programming.  For  this  reastm, 
backgrounds  and  argumentation  receive  cmly  a  brief  discussion.  Subjects  that  do  receive  indeplh 
discussion  are:  the  definition  of  range-restricted,  the  several  shapes  that  range-forms  can  assume 
and  the  simpliiication  algorithm  that  enables  (more)  efficient  integrity  checking.  The  definition  of 
range-restriaed  as  well  as  the  simplification  algorithm  are  generalizations  of  an  a^^roach  to 
integrity  checking  in  ordinary  databases,  as  is  proposed  in  [Nicolas82}. 

Definition : 

Let  FsQM  be  a  formula  disjunctive  miiumum  form,  thus  a  disjunction  of  conjunctions,  where  Q 
is  the  coiKatenation  of  quantifications  (Vx3yVz...etc)  and  M  a  quantor-free  formula. 

1.  If  X  is  an  existentially  quantified  vuiable  in  F  (in  other  words,  3  occurs  in  (^,  dien  F  is 
range-restricted  in  relation  to  X  if  every  conjunction  of  M  with  X  in  it,  coittains  at  least  1 
positive  X-literal. 

2.  If  is  X  uruversally  quantified  in  F  (in  other  words,  V  occurs  in  Q  ).  then  F  is  range- 
restricted  in  relaikm  to  X  if  the  diqunctive  minimum  form  of  -i  F  is  range-restricted  hi 
relation  to  X. 
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3.  F  is  range-restricted  if  F  is  range-restricted  in  relation  to  all  variables  in  Q. 

(An  X  literal  is  an  atom  containing  X,  e.g.  p(X)  or  r(X,\').  or  its  negation). 

The  essence  of  this  definition  that  is  probably  hard  to  scrutinize,  may  become  clearer  with  help  of 
the  example  below. 

Example: 

Suppose  one  wishes  to  include  a  constraint  in  a  database  saying  that  the  defence  staff  breaks 
down  into  civil  personnel  (b)  and  military  personnel  (m),  with  no  personnel  being  member  of  both 
denominators.  First-order  logic  can  formulate  this  constraint  as  follows: 

VX(  (b(X)  V  m(X))  A  -n  (b(X)  A  m(X)) )  (1) 

This  formula,  however,  is  not  range-restricted.  A  disjunctive  minimum  fonn  of  it  is: 

F  =  VX(b(X)A^m(X)v(m(X)A-,b(X)))  (2) 

In  terms  of  the  definition  of  range-restricted,  this  formula  relates  to  case  (2)  and  is  F  equivalent 
to: 


3X(  b(X)  A  m(X)  V  b(X)  A  -1  m(X) )  (3) 

This  fomiula  is  not  range-restricted  because  the  conjunction  -■  b(X)  a  -■  m(X)  does  not  ctmtain  a 
positive  X-literal.  Should  one  attempt  to  represent  this  fonnula  in  Prolog  and  encounter 

not  ( (b(X),  m(X)) ;  (not  b(X),  not  m(X)) )  (4) 

then  the  integrity  check  in  die  daudiase  below  comes  to  a  standstill.  Evaluation  of  (4)  yields  the 
result  'yes',  but  without  die  specification  whether  Joe  belongs  to  either  civil  or  military  personnel. 


L 
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function ( jim, quartermaster) , 
function ( john, technician) . 
function ( joe, chauffeur) . 
military ( jim) . 
civilian ( john) . 


Figure  6. 1 :  Database  belonging  to  the  example. 

The  faa  that  the  integrity  check  fails  is  due  to  the  semantics  of  the  not-operator  in  Prolog,  in  other 
words  the  negaticm-as-failure  rule.  The  Prolog  expression  (4)  is  in  fact  not  the  translation  of  the 
first-order  fomiula  (3),  but  of: 

-.(3X(b(X)Am(X))v(-.3Xb(X)A^3Xm(X))]  (5) 

To  avoid  such  subtle  mistakes  in  the  use  of  the  not-operator  in  Prolog,  one  can  use  the  rule  of 
thumb  that  all  variables  in  the  argument  of  the  not-operator  have  to  be  instantiated  (ground  terms) 
before  'not'  is  executed.  This  is  exactly  what  is  guaranteed  by  the  notion  of  range-restricted. 

If,  by  insertion  of  the  predicate  p  (of  personnel)  instead  of  (4); 

not  (p(X),  ( (b(X),  m(X)) ;  (not  b(X).  not  m(X)) ) )  (6) 

is  used,  the  integrity  check  will  run  satisfactory.  This  Prolog  expression  correspraids  with  the 
range-restriaed  formula: 

VX(p(X)~»(b(X)A-.m(X)vm(X)A-,b(X)))  (7) 

where  p(X)  is  called  the  range-expression. 

Especially  experienced  Prolog  programmers  will  ask  themselves  why  everything  is  so 
complicated  (using  rvige-restricted  formulas)  when  simidicity  lives  nextdoor  (using  ground 
expressions  in  Prcdog).  It  is  true  that  the  notion  of  range-restricted  can  be  by-passed  and  tme  even 
does  not  need  Decker's  meta-iruerpreter  (to  be  discussed  below)  for  the  evaluation  of  constrairtts 
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when  using  Quintus  Prolog.  Range-restricted  constraints  can  be  included  as  ordinary  Prolog 
goals,  provided  that  one  has  operators  for  and,  or  and  not  at  one's  disposal  (as,  for  instance,  in 
Quintus  Prolog).  Nevertheless,  reading  aitd  formulating  constraints  is  enhanced  when  these  can  be 
formulated  in  a  predicate  logic  form.  The  transition  of  first-order  (range-restricted)  expressions  to 
range  forms  that  can  be  evaluated  or  even  Prolog  goals  can,  in  princii^e,  be  automated.  The 
notion  of  range-restricted  therefore,  brings  us  closer  to  declarative  programming  and  as  such,  can 
be  worthwhile  for  the  quality  of  software  for  expert  systems. 

Decker  proves  that  all  range-restricted  formulas  can  be  transcribed  into  the  following  standard 
fotm(s).  Either: 

1 .  F  only  contains  existentially  quantified  variables  (with  every  conjunction  containing  a  Y, 
contains  at  least  one  positive  Y-liteial),  or, 

2.  F  has  the  form  of  3X(P  a  F)  v  P,  with  P  being  a  so-called  range-expression  in  X  (a 
purely  existentially  quantified  disjurKtion  of  positive  X-liierals)  and  F  and  F'  range- 
restricted  formulas  in  standard  form,  or, 

3.  F  has  the  form  of  VX(P  -»  F)  a  F',  with  again  P  being  a  range-expression  in  X  and  F 
and  F'  range-restricted  fonnulas  in  standard  form. 

Furthermore,  Decker  states  that  the  problems  as  far  as  the  evaluation  of  a  range  form  in  Prolog  are 
concerned,  are  limited  to  version  3  of  the  range  form.  If  the  interpreter  can  handle  range  forms 
formed  as  VX(P  -» F),  it  can  evaluate  all  range  forms. 

After  all,  the  existential  quamor  is  implicitly  present  for  all  variables  in  Prolog  clauses.  By  using 
the  auxiliary  predicate  'forall'  (in  the  shape  of  forall(X,Xrex,CoiKl)  as  the  translation  of  VX(Xrex 
CotKl) )  and  to  write  a  meta-inter^ter  (see  figure  6.2)  that  can  process  this  predicate  and 
directs  all  expressions  dut  occur  in  constraints  diat  do  not  coiuain  forall  in  the  regular  Prolog 
interpreter,  the  {Moblem  can  be  solved. 

For  clarity's  sake,  it  must  be  rrated  diat  the  first  argument  for  die  forall  predicate  may  be  omitted 
here.  It  has  only  been  added  for  the  sake  of  readability,  not  for  the  sake  of  evaluation.  The 
algoridan  supposes  in  fact  that  Xrex  is  a  range-eiqxession  in  X.  Should  diis  not  be  the  case  then 
the  corsdaim  will  have  to  be  fumulaied  differently. 


TNO  report 


Page 

65 


evaluate (Constraint) verify (Constraint) . 

verify(forall(X,Xrex,Concl)  ) :- 
call (Xrex) , 

falsify (Concl) , ! ,fail. 
verify(  forall (X, Xrex, Concl)  ):-  !. 
verify (Constraint) call (Constraint) . 
verify (Constraint ) : -  ! , fail . 

falsify (Constraint) verify (Constraint ) , ! , fail . 
falsify  (Constraint) . 


Figure  6.2:  Prolog  meta-interpreter. 

As  touched  upon  before  in  the  discussion  of  the  notion  of  range-restricted,  the  meta-interpreter 
may  be  forgotten  if  one  uses  the  Prolog  expression  "NOT  (Xrex,  not  Concl)’  instead  of  'forall  (X, 
Xrex,  Concl)'. 

The  simplification  of  constraints  in  updates  of  ordinary  databases  is  quite  simple.  In  the  case  of 
an  INSERT,  true  is  substituted  in  the  constraints  in  those  places  where  the  fact  to  be  inserted 
occurs.  In  case  of  DELETE,  a  false  is  substituted.  This  needs  to  be  done  only  once  (compile 
time). 

Example'. 

From  the  constraint; 

VX(  -1  persotmel  (X)  v 

(military  (X)  a  -i  civilian  (X))  v 
(civilian  (X)  a  -•  military  (X))  ) 


the  following  (com|Me-tiine)  updtie-conditions  ait  derived  for  the  various  kinds  of  updates: 
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INSERT  personnel(X)  ONLY-IF  !nilitaiy(X)  a  civilian(X)  v 
civilian(X)  a  -•  militaryCX) 

INSERT  mUitary(X)  ONLY-IF  personneI(X)  v  not  civilian(X) 

INSERT  civiIian(X)  ONLY-IF  -i  personnel(X)  v  not  military(X) 

DELETE  militaryCX)  ONLY-IF  -•  personnel(X)  v  civilian(X) 

DELETE  civilian(X)  ONLY-IF  -» personnel(X)  v  militaryCX) 

When,  at  this  point,  an  actual  update  takes  place,  for  instance,  INSERT  civilian(john),  only  -• 
personnelCjohn)  v  militaryCjohn)  has  to  be  evaluated. 

While  checking  the  integrity  of  deductive  databases,  not  only  the  update  itself  has  to  be 
determined  but  also  its  consequences  Cin  terms  of  derivable  facts)  in  order  to  locate  the  constraints 
that  must  be  evaluated.  Actually,  two  sets  of  facts  must  be  calculated,  i.e.  the  set  of  facts  that  can 
be  derived  after  the  update  has  taken  place  and  which  could  not  be  derived  before  that,  and  the  set 
of  facts  that  originally  could  be  derived  before  the  update  but  cannot  afterwards.  The  sets  are 
indicated  with  A4-  and  A-.  In  this  respect  it  is  important  to  realize  that  adding  a  fact  to  A+  as  well 
as  A-  can  contribute  to  the  presence  of  'not'  in  derivation  rules. 

The  same  goes  for  the  deletion  of  a  faa.  Decker  first  describes  how  the  sets  of  A-t-  and  A-  could 
be  calculated  straightforwardly,  and  then  presents  a  (more)  efficient  procedure  for  determining  B+ 
and  B-  as  estimators  of  A+  and  A-  with  A-r-  c  B+  en  A-  c  B-.  Decker  relies  on  the  reader's  own 
alertness  to  note  that  B-r-  and  B-  are  not  obligatorily  equivalent  to  A-«-  and  A-. 

The  straightforward  method  consists  of  direct  application  of  the  definitions  af  A-i-  and  A-.  Let  LB 
be  the  database  before  the  update  and  DBN  after  the  update,  DB*  the  extensicm  of  DB  and  DBN* 
that  of  DBN.  These  exumsions  are  defined  as  the  merger  of  the  explicit  facts  with  the  implicit 
ones  (derivable  form  the  derivation  rules  of  the  database).  As  Head:-Body  is  a  derivation  rule  then 
set_of(Head.Body,Seii)  yields  the  set  Set  of  all  instances  of  Head  that  are  a  memeber  of  the 
extension  and  can  be  derived  using  the  appit^ate  derivation  rule.  By  merging  ail  these  sets  with 
explicit  facts  the  comfrieie  extenskm  can  be  obtained.  Subsequently,  A+  and  A-  can  be  calculated 
from  DB*  aiKl  DBN*  via  A-f  *  DBN*SDB*  and  A- «  DB*\DBN*.  Of  cmirse,  calculating  A-i-  and 
A-  in  this  mmner  is  not  very  sennlrle.  It  is  so  time-consuming  that  simplification  of  the 
consuatnts  would  yield  i  nett  loss  of  time  rather  dum  gaining  iL  The  more  efBciem  method 
suggsied  by  Dedcer  comes  down  to  introducing  an  elemem  of  chance  and  exchanging  space 
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complexity  against  time  complexity:  of  every  derivation  rule  in  the  database  a  track  is  made  of 
which  literals  can  contribute  to  its  conclusion,  either  in  a  positive  or  negative  sense.  See  figure  6.3 
for  an  example. 


p(X) q{X),r(X),not  s {X) . 

occurs_positive {q(X) ,  {p(X) q (X) , r (X) , not  s (X) )  ). 
occurs_positive (r (X) ,  (p{X) q (X) , r (X) , not  s (X) )  ). 
occurs_negative (s (X) ,  (p(X) q (X) , r (X) , not  s(X))  ). 


Figure  6.3:  Transformation  of  a  Prolog-rule  according  to  Decker. 

Now,  it  is  no  longer  necessary  to  determine  the  extension  of  aU  derivation  rules  in  DB  and  DBN. 
Instead,  only  the  extensions  of  the  derivation  rules  in  the  update  will  be  detennined  and  these  give 
a  first  contribution  to  B+  and  B-  (together  with  the  facts  in  the  update). 

Next,  the  transitive  closures  of  B+  and  B-  are  determined  using  the  meta-information  about  die 
rules,  as  defined  in  the  'occurs'-facts. 

If  that  is  the  case,  the  extension  of  the  corresponding  instantiation  of  the  second  argument  is 
added  to  B-t-  and  B-  (if  a  fact  from  B-i-  can  be  unified  with  a  literal  in  an  occurs-negative  fact  to  B- 
,  if  a  B-  fact  with  an  occurs-positive-literal  can  be  unified  with  B-  ...  etc.).  In  fact,  after  every 
addition  B-f  or  B-,  there  is  an  immediate  inspection  whether  the  added  element  can  be  unified  to  a 
literal  in  an  update  constraiiu.  If  that  is  the  case,  the  corresponding  (possibly  instantiated) 
condition  is  checked  immediately.  This  jumping  to-and-fto  (interleaving)  betweem  the  expansion 
of  B-f  and  B-  and  dK  actual  integrity  check  increases  the  chances  of  fast  discovery  of  a 
transgression  against  integrity.  After  all,  the  elements  added  to  B-f  and  B-  through  the  occurs- 
predicates  become  more  guessable  as  the  number  of  occurs-facts  used  increases. 
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6.3  Shapiro's  algorithm 

We  have  already  touched  upon  the  faa  that  Shapiro's  algorithm  starts  with  a  number  of 
assumptions  concerning  the  fomiaiism  on  which  the  debugging-algoritms  are  applied 
[Shiy}iro84].  This  formalism,  i.e.  a  program  in  a  certain  programming  language,  can  be  seen  as  a 
reflection  of  the  specification  of  a  proUem-domain.  Firstly,  the  basic  element  of  this  language  is 
the  procedure  (or  function).  During  the  debugging  process  only  the  calls  of  these  procedures  are 
of  importance,  together  with  input  and  output.  Secondly,  every  procedure  has  a  name,  arity  (the 
number  of  arguments)  and  the  corresponding  code.  Thirdly,  the  program  code  registers  a  set  of 
actions.  Fourthly,  all  calculations  of  the  program,  i.e.  series  of  subsequent  procedure  calls,  can  be 
described  with  a  tree-structure  that  contains  the  calls  in  its  nodes  and  points  of  succes  or  failure  in 
its  leaves. 

Shapiro's  algorithm  was  primarily  aimed  at  program  debugging;  the  location  and  correction  of 
errors  in  program  code  [Hamdan89].  These  errors  can  have  different  causes,  such  as  inadequate 
functional  design  or  a  failing  system  development  method.  According  to  Shapiro,  however,  a 
mote  fundamental  cause  can  be  pointed  out.  He  sees  a  program  as  the  epitomization  of  a  complex 
set  of  assumptions.  The  behaviour  of  a  program  is  a  derivation  of  these  assumptions,  which  makes 
it  difficult  to  predict  its  behaviour  completely.  The  considerations  were  an  incentive  for  Shiq>iro 
to  develop  a  theory  that  must  produce  an  answer  to  two  questions: 

•  how  can  an  error  be  detected  in  a  program  that  does  not  function  properly?; 

•  how  can  this  error  be  corrected?. 

This  theory  has  led  to  the  construction  of  algoritms  for  each  of  these  two  fHoblems.  With  the  help 
of  a  diagnosis-algorithm,  errors  must  be  detected,  after  which  improvements  can  be  made  using 
the  error-correction  algorithm.  These  two  algorithms  are  both  included  in  a  so-called  'Model 
Inference  System'  (MIS)  which  takes  a  program  that  must  be  debugged  for  its  irgrut  and  a  list  of 
input  /  output  examples,  that  partly  determines  the  behaviour  of  the  program. 

It  is  necessary  that  during  the  debugging  process  it  is  dear  what  the  expected  bdiaviour  of  the 
program  will  be.  It  is  always  assumed  that  there  is  an  authority  tlutt  can  give  an  ansvrer  to  the 
questions  that  are  posed  by  the  system  dx)ut  the  Universe  of  Discourse'.  The  answer  can  be  given 
by  the  developer  of  the  program,  or  by  another  program.  When,  fbr  instance,  a  propoly 
functioning  version  of  a  program  will  be  altered,  questions  that  are  generated  during  debugging 
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can  be  'answered'  by  the  old  version.  It  is  even  possible  to  build  a  (simple)  program  from  scratch 
by  starting  with  an  'empty'  program.  MIS  as  well  as  the  programs  to  be  debugged  are  written  in 
Prolog. 

The  integrity  check  of  knowledgebases  is  chiefly  concerned  with  detecting  errors  in  'executable 
speciflcations'.  It  is  assumed  here  that  the  specification  of  a  knowledge-domain  has  been 
established  with  the  help  of  ENIAM.  The  resulting  conceptual  model  can  be  transformed  into  a 
formal  representation  in  Prolog  that  can  be  executed  on  the  computer.  This  reprot  pays  special 
attention  to  what  is  called  non-graphic  constraints  in  traditional  NIAM.  The  rules  form  an 
important  part  of  the  knowledge  in  a  knowledgebase  and  may  be  compared  to  production  rules. 
The  diagnostic  part  of  Shapiro's  algorithm  can  check  the  integrity  of  this  Prolog  code. 

By  applying  Shapiro's  algorithm  to  'pure'  Prolog  programs,  three  types  of  errors  can  be  detected; 
letmination  of  a  program  with  incorrect  results,  termination  with  missing  results  and  non¬ 
termination.  The  basis  of  the  diagnostic  algorithm  is  formed  by  a  meta-interpreter,  which  is  an 
interpreter  for  a  programming  language  written  in  that  same  language.  The  meta-interpreter 
enables  the  user  to  subject  the  calculation  process  of  a  program  to  further  scrutiny.  As  such  it  may 
be  compared  with  the  trace  facility  that  is  used  in  languages  such  as  C,  Pascal,  Prolog  and  Lisp.  A 
meta-interpreter  is  a  meta-program  that  can  treat  other  programs  as  data  [SterlingSb],  which 
simplifies  their  manipulation,  analysis  and  simulation.  At  this  point,  an  instructional  example  of  a 
meta- interpreter  for  'pure'  Prolog  (i.e.  simple  Prolog  without  operators  such  as  'cut'  and  'not',  for 
example)  is  in  order: 

This  meta-inteipeter  keeps  a  record  in  the  form  of  a  proof-tree  of  every  proof  that  must  be  given. 
Briefly,  its  operation  is  as  follows.  The  atom  'true'  is  always  true.  The  conjuncticHi  of  two  goals  is 
true  if  they  both  are  true.  A  goal  containing  a  body  with  subgoals  is  true  if  all  subgoals  are  true. 
The  last  solve/2-line  of  figure  6.4  conuins  the  [xedicate  'clause(A,B)'.  This  separates  the  head  and 
body  of  a  line;  B  contains  the  conjunction  of  subgoals  of  a  goal  named  A.  Taking  as  a  rule  'A;- 
®1’  ®2*  ®3’  ®4-'*  second  argument  of  solve/2  assumes  the  form:  (B|,  (B2,  (B3,  B4))).  To  be 
able  to  debug  a  program,  the  developer  must  have  an  idea  of  the  program's  behaviour  when 
applied  to  a  certain  field  of  application.  With  the  help  of  a  meta-interpreter  this  program  can  now 
be  simulated.  Using  the  debugging  algorithm,  it  will  then  be  attempted  to  detect  a  difference 
between  the  expected  and  the  actual  behaviour. 
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solve (true, true) . 
solve ( (A, B> , (Proof A, Proof B) ) : - 
solve (A, Proof A) , 
solve (B, ProofB) . 
solve (A, (A: -Proof ) ) : - 
clause (A, B) , 
solve (B, Proof) . 


Figure  6.4:  Meta-interpreter  for  simple  Prolog. 

Before  giving  a  more  precise  description  of  Sht^iro's  algorithms,  it  might  be  clarifying  to  say  a 
few  words  about  the  semantics  of  logic  programs'.  Prolog  in  this  case  [Shapiro84;Sterling87]. 

The  clause  'A:-  contains  A'  in  the  interpretation  M  (i,e.  Meaning)  if  a  substitution  6 

exists  unifying  A  with  A'  and  makes  the  goals  B|,B2,...,Bn  in  M  true.  A  clause  is  'true'  in  M  if 
every  variable-free  goal  of  this  clause  is  contained  in  M,  and  'false'  if  not.  This  assertion  can 
easily  be  reflected  in  Prolog.  In  this  environment,  facts  are  true  when  they  find  themselves  in  the 
internal  database,  or  when  they  can  be  derived  by  applying  a  rule.  A  program  P  in  M  is  true  if 
every  clause  in  P  is  true  in  M.  The  notion  'true'  may  in  a  logical  context  be  taken  as  ‘correcf, 
'false'  as  'not  correct'. 

The  interpretation  (meaning)  of  a  program  P  is  M(P),  which  corresponds  with  the  set  of  variable- 
free  (fully  instantiated)  goals  in  P  that  are  true.  Now  we  can  say  that  a  program  P  is  complete  in 
M  if  M  c  M(P).  A  program  P  is  only  then  complete  as  well  as  correct  in  M  if  M  =  MfP). 

The  drxnain  D  of  a  program  is  the  set  of  vari^le-free  goals  of  P.  A  program  P  terminates  on  a 
domain  D  if  there  is  not  a  singte  goal  A  in  that  leads  to  an  infinite  derivation  of  goal  A  in  P. 

Previously  three  types  of  errors  were  described  dtat  can  be  detected  with  the  help  of  Shapiro's 
algorithm.  The  basis  of  this  is  always  the  'proof-tree'  that  is  built  during  the  execution  of  the 
program.  This  may  also  be  represettfed  by  a  'top-level  trace'  of  a  (Hocedure  (prediutte  in  Prolog)  p 
with  X  as  input  and  y  as  output.  This  trace  ctmsists  of  a  (possiUy  empty)  set  of  triplets  in  the  form 
of  {  <P|.  X|.  y|>,  <P2.  X2.  y2> . <Pii.  )•  If  the  set  is  cmpQr,  then  the  procedure  p  with 
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inpit  X  can  yield  the  ouQwt  y,  without  procedure  calls.  If  the  set  is  not  empty,  then  a  chain  of 
procedure  calls  with  x  as  its  initial  input,  will  eventually  yield  an  output  y. 

A  proof-tree  is  complete  if  all  imdes  are  triplets  with  the  fonn  <p,  x,  y>  and  all  leaves  have  the 
empty  set  as  'top-level  trace'.  It  can  be  said  diat  y  is  the  correct  output  of  the  procedure  call  <p,  x> 
in  M,  if  <p,  X,  y>  is  contained  in  M. 

A  program  P  is  correct  in  M  if  the  root  of  every  complete  'proof  tree'  of  P  is  contained  in  M.  A 
program  P  is  complete  in  A/  if  every  triplet  in  M  is  the  root  of  a  complete  'proof  tree'  of  P.  A 
program  P  is  terminating  if  the  'proof  tree'  contains  no  infinite  path. 

Now  we  will  demonstrate  an  example  of  an  algorithm  for  detecting  an  incorrea  solution,  based 
on  [Shapiro84;Sterling87].  A  program  P  can  only  yield  an  incorrect  solution  if  it  contains  an 
incorrect  clause.  A  clause  C  is  not  correct  within  the  inteipretation  M  if  is  an  instantiation  of 
the  body  of  C  that  is  true  in  Af  and  a  head  that  is  not  true  in  M.  The  description  is  as  follows: 

•  Input:  procedure  p  in  P  and  an  input  x,  such  that  <p,  x>  yields  an  output  y  that  is  not 
correct  in  M. 

•  Output:  a  triplet  <q,  u,  v>  not  in  M. 

•  Algorithm:  simulate  the  execution  of  p  on  input  x  with  y  as  result;  if  a  call  <q,  u>  leads  to 
the  output  V,  establish  (by  letting  the  user  pose  a  question)  if  <q,  u,  v>  is  contained  in  M. 
If  this  is  not  the  case  then  yield  the  incorrect  solution  <q,  u,  v>  and  stop. 

It  is  hard  to  determine  whether  a  program  will  not  terminate,  i.e.  run  indefinitely.  Shapiro  opts  for 
a  pragmatic  solution  to  this  problem  by  providing  the  meta-interpreter  a  preset  limit  to  the  depth 
of  recursion.  One  of  the  ai;guments  contains  a  proof  tree  of  procedure  call  upto  the  moment  of 
termination.  By  examining  this  trace  further,  diagnosis  of  the  problem  can  take  place.  Beside 
errors  in  the  program,  the  cause  may  also  be  found  in  a  lack  of  memory  capacity,  being 
responsible  for  a  stack  overflow.  The  description  of  the  algorithm  for  detecting  non-termination  is 
as  follows  [Shapiro84;Sterling87]: 

•  Inimt:  procedure  p  in  P,  an  input  x  and  an  integer  d  >  0,  such  that  the  call  <p,  x>  will  not 
exceed  the  maximum  depth  of  recurskm  d. 
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•  Ou^t:  when  exceeding  d,  y  will  contain  the  set  of  procedure  calls  running  the  length  of 
d,  consisting  of  triplets  with  the  form  <p,  x.  y>. 

•  Algorithm:  simulate  the  execution  of  p  on  x;  if  depth  d  has  been  reached  without  iinding 
a  solution,  then  the  program  will  not  terminate. 

Of  the  three  types  of  errors  that  can  be  found  with  Shapiro's  algorithm,  detecting  a  missing 
solution  is  the  hardest  one  to  solve.  The  notion  of  'cover'  plays  an  important  role  here.  A  certain 
clause  C  is  a  cover  of  goal  A  in  relation  to  intertaetation  Af,  if  there  is  an  instantiation  of  C  of 
which  the  head  corresponds  with  A  and  the  body  is  contained  in  Af.  If  a  program  P  has  a  missing 
solution  in  interpretation  Af,  then  there  is  a  goal  A  in  A/  that  has  no  cover  in  a  clause  of  P.  During 
the  operation  of  the  algorithm  below,  the  user  must  answer  questions  about  the  validity  of  certain 
instantiations. 

A  program  P  is  complete  in  M  if  for  every  triplet  <p,  x,  y>  in  Af  the  call  <p,  x>  yields  the  output 
y.  This  means  that  p  must  be  a  cover  for  <p,  x,  y>,  if  it  is  not,  then  we  have  of  a  missing  solution. 
The  remedy  then,  is  to  change  the  procedure  p,  such  that  y  can  be  a  result  The  algorithm  for 
detecting  a  missing  solution  is  as  follows  [Shapiro84;Sterling87]: 

•  Input:  procedure  p  in  P,  an  input  x  and  an  output  y  all  contained  in  M,  which  make  p  fail. 

•  Output;  a  triplet  <q,  u,  v>  in  M,  for  which  q  is  no  cover. 

•  Algorithr.'.;  simulate  the  execution  of  p  (xi  x  with  y  as  a  result,  using  the  existential 
queries  and  simultaneously  keeping  track  using  a  proof  tree;  in  case  of  a  'fail'  the  triplet 
<q,  u,  v>  is  produced  and  the  algorithm  will  stop. 

6.4  Nguyen's  algorithm 

Nguyen  has  developed  two  algorithms,  elaborating  on  the  conce{)ts  introduced  in  [Suwa82],  to 
check  the  consistency  and  completeness.  The  CHECK-algorithm  [Nguyen8SJ<lguyen87b]  is 
closest  to  the  Suwa-approach.  Besides  contradiction,  redundancy  and  subsumption,  under  the 
denominator  consistency  CHECK  also  searches  for  unnecessary  conditions  and  circular  rule- 
chains,  for  the  sake  of  consistency. 
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An  example  of  unnecessary  conditions.  The  rules: 

IFp=  1  ANDq»tlTHENAenIFp?tl  ANDq^tlTHENA 
may  be  merged  into: 

IFqstlTHENA. 

The  fact  that  circular  rule-chains  are  checked  already  indicates  that  CHECK  not  only  looks  at  flat 
(parts  oO  rule-files.  This  is  also  reflected  in  the  checking  of  completeness.  Beside  conditions  that 
ate  not  covered  by  parameter  values,  a  search  is  also  made  for  urueachable  conditions,  dead-end 
conditions  and  dead-end  goals. 

The  CHECK-algorithm  is  suitable  for  diagnosing  knowledgebases  made  with  the  help  of  a 
'generic  expert  system  shell'  (Lockheed  Expert  System  [LaffeySb]  in  this  case).  The  rules  in  LES 
assume  the  fonn  IF<conjunction  of  conditions>THEN<conjunction  of  conclusions>.  Beside 
backward-chaining,  LES  also  provides  for  forward-chaining.  The  discussion  here  is  limited  to 
backward-chaining,  because  there  is  not  much  difference  in  integrity-checking  in  forward  arxl 
backward-chaining.  Only  the  notion  of  unreachable  conclusions  is  meaningless  in  forward- 
chaining. 

Like  ONCOCIN,  LES  uses  a  kind  of  context-mechanism.  Anyhow,  Nguyen  only  ^lies  an 
integrity-check  to  component  parts  of  the  whole  rule-file  of  an  apfdication,  thus  substantially 
reducing  the  computational  costs  of  the  algorithm.  Furthermore,  LES-rules  can  contain  variables. 

[NguyenSS]  gives  a  description  of  the  algorithm  in  an  Algol-like  notation.  However,  this  is  so 
incomplete  that  it  is  not  clear  how  a  consistency-check  is  performed,  let  alone  the  possibility  of 
making  a  statement  about  its  efficiency:  declarations  of  variables  are  not  included  and  the  result- 
type  of  the  function  Compare.clauses  (..., ...)  must  be  guessed.  The  futKtion  itsdf  has  not  been 
worked  out  The  same  goes  for  the  Transform  function  in  the  allocation  'matched.rule  « 
Transform  (i,  k,  clause-relations)'. 

[NguyenSTb]  verbally  (i.e.  not  Algol-like)  describes  how  the  algorithm  basically  works.  Of 
course,  things  are  associated  with  the  working  method  of  LES.  The  main  issues  are  as  follows. 
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Nguyen  lemaiks  that  in  LES,  multiple  goals  can  be  'active',  with  one  or  more  rule-sets  being 
coufded  to  each  goal.  A  rule-set  contains  pitxluction  rules  that  can  contribute  to  accomplishing  the 
goals  involved.  Nguyen  also  notes  that  the  rule-sets  of  the  goals  may  be  tested  separately  and  it 
does  not  form  a  significant  limitation  for  the  type  of  production  rule  systmns  for  which  the 
algorithm  can  be  used.  In  case  of  a  possible  unwanted  interaction  or  depmKiability  between  the 
rules  in  different  sets,  these  sets  are  merged  into  a  new,  larger  set  Please  consider  the  following. 
A  rule-set  consists  of  a  number  of  rules.  Every  rule  contains  an  IF-part  and  a  THEN-part  The  IF- 
part  contains  one  or  more  conditions,  the  THEN-part  one  or  more  conclusions. 

It  is  not  clear  what  the  individual  conclusions  and  conditions  exactly  look  like  (assuming  that  IF 
?X  has  a  fever,  AND  ?X  has  flat  pink  spots  on  his  skin  THEN  type  of  disease  of  ?X  is  measles')  is 
a  'natural  language'  representation  of  'IF  symptom(?X,  fever)  AND  symptom(?X, 
flat_pink_spots_on_skin)  THEN  disease.type  (?X,  measles)'  or  something  similar.  In  the  latter 
form,  the  IF-part  thus  contains  two  conditions,  i.e.  'symptom  (?X,  fever)'  and  'symptom  (?X,  flat- 
pink-  spots-on-skin)'.  Nguyen  (1987b]  now  sketches  the  following  procedure:  First,  all  individual 
conditions  and  conclusions  are  compared  (in  pairs).  The  result  is  a  table  with  for  every  pair  n  of 
the  titles  SAME.  DIFFERENT,  CONFLICT,  SUBSET  or  SUPERSET.  From  these  clause-clause 
relationships  (Nguyen  speaks  about  conditions/conclusions  and  about  EF-clauses/THEN-clauses, 
interchangeably)  the  relationships  between  the  IF-  and  THEN-parts  of  the  various  rules  are 
produced.  Rnally,  these  are  used  to  indicate  whether  pairs  of  rules  are  conflicting,  subsuming, 
redundant  etc. 

Beside  the  CHECK-algorithm,  Nguyen  also  describes  the  ARC-algorithm.  ARC  stands  for  ART 
Rule  Checker  [Nguyen87a].  According  to  Nguyen,  this  offers  additional  possibilities  in  the  form 
of  detecting  redundant  nlt-cHains,  subsumed  nile-chains  aixl  conflicting  nile-ckains.  Nguyen 
does  not  describe  how  these  additions  are  realized.  As  said  before  (in  other  teims),  the  three 
articles  that  were  studied  (NguyenSS,  Nguyen87a,  NguymSTb],  would  very  well  have  lend 
themselves  for  a  'text-checker'  in  the  Sfririt  of  Nguyen  himself.  They  ccmtain  a  lot  of 
inconsistencies  and,  what  is  more  importarn,  ate  very  incomplete.  On  the  basis  of  the  information 
pven  above,  the  description  of  Puuronen's  algorithm  and  tme's  own  ideas,  it  is  inobaUy  far  more 
effective  to  build  a  rule-checker  from  scratch  than  to  attempt  to  reconstruct  Nguyen's  algorithm. 
Therefore,  let  us  conclude  this  discussitm  about  ctmsistency  and  completeness,  the  Nguyen  way, 
with  stating  our  surmise  about  the  limited  possibilities  for  diecking  that  are  offered  by  die 
algorittans,  as  well  as  our  grounds  for  dds  surmise. 
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Surmise:  ARC  only  detects  flagrant  errors.  The  assertion  that  ARC  is  able  to  detect  conflicting 
and  redundant  nile-chains,  is  misleading.  Frtxn  [Nguyen87a]  it  can  be  deduced  that  ARC  can  only 
do  this  when  the  flist  rules  of  the  two  chains  have  the  same  or  equivalent  IF-parts.  Two  IF-paits 
are  equivalent  if  they  contain  the  same  number  of  conditions  and  these  conditions  are  equivalent 
pairwise.  Two  conditions  are  equivalent  if  they  can  be  unified.  All  this  implies  that  the  conflict  in 
the  rules  below  cannot  be  detected  by  ARC: 

Example: 

IF  A=1  THEN  B=1 

IF  A=1  and  C=1  THEN  0=1 
IF  B=1  and  C=1  THEN  D=0 


The  saine  goes  for  subsumption.  And  what  holds  true  for  ARC,  holds  especially  true  for  CHECK. 

The  di.'^cussion  above  does  not  mean  that  CHECK  (or  ARC)  is  a  botch.  On  the  one  hand  an 
algoriihm  that  performs  a  more  thorough  check  is  highly  desirable  from  a  quality  assurance  point 
of  view,  on  the  other  hand  we  must  not  forget  the  price  tag  (in  terms  of  complexity). 


6.5  A  comparative  analysis 

In  tlK  category  of  production-rule-oriented  methods,  Nguyen's  algorithms  perform  the  most 
extensive  integrity  checks.  He  goes  beyond  checking  one-step  inference  aixl  also  looks  at  the 
weU-fc'rmedness  of  rule-cAoins.  It  must  be  said,  though,  that  the  general  character  of  this  rule- 
chain  check  is  somewhat  disappointing.  Redundant  and  conflicting  rule-chains  are,  for  instance, 
only  detected  when  the  first  rules  of  both  diains  comain  identical  or  equivalent  IF-parts.  This  type 
of  redundancy  c.q.  conflict  is  the  most  flagrant  and  thus  will  be  the  easiest  one  to  prevent  by 
programming  more  carefully.  Nguyen's  consistency  dieck  too,  does  not  qualify  as  being 
complete. 

When  you  compare  the  ctmcepts  of  consistency  and  completeness  with  the  concqM  of  integrity  of 
deductive  databases,  dien  subsumption  i^tpears  to  be  out  of  the  question:  bmh  explaruoions  of 
integrity  contain  elements  not  mutually  shared.  Of  the  various  facets  of  productkm-nde 
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consistency,  the  database-oriented  t^^xoach  covers  only  one  (conflict),  but  this  hai^mis  in  a  much 
wider  sense  because  the  latter  iqjproadi  enables  the  knowledge  mgineer  to  impose  semantic 
restrictions  to  the  database  in  a  general  soise.  Furthermore,  it  is  more  thorough-  Every  conflia 
that  can  be  derived  will  actually  be  detected.  This  makes  die  faa  diat  subsumption  and 
redundaiKy  are  not  detectable  less  serious.  After  all,  they  are  not  imminently  disastrous  for  the 
relialxlity  of  the  expert  system.  After  maintenance  (dianges  in  the  knowledgebase)  they  can  be  a 
source  of  contradictions,  but  as  soon  as  an  intended  update  thieatens  to  cause  diis,  this  will  most 
certainly  be  detected  during  an  integrity  dieck.  The  absence  of  a  circularity  check  is  more  serious. 
In  a  certain  (ironic)  way,  circularity  is  detected  because  the  integrity  check  comes  to  a  standstill. 
The  complete  lack  of  a  check  for  completeness  in  the  deductive  ^roach  too,  is  a  ftmdamoital 
flaw  in  this  method. 

Shapiro's  algorithm  is  designed  to  detect  three  possible  errors  in  programs:  non-termination, 
incorrect  solutions  and  missing  solutions.  Considering  the  descriptions  of  consistency  and 
completeness  it  can  be  said  that  Shapiro's  interpretation  of  these  notions  covers  various  aqiects. 
As  far  as  consistency  is  concerned,  the  conflict  and  circularity  are  a^cts  analysed  with  the  help 
of  algorithms  for  the  detection  of  incorrect  solutions  and  non-termination.  The  algorithm  for 
detecting  a  missing  solution  indicates  where  there  is  a  missing  line  in  the  program  that  should 
generate  this  solution;  this  covers  the  completeness  aspea. 

Non-termination  is  detected  by  Shapiro  when  a  preset  maximum  depth  of  recursion  is  reached  by 
the  program,  without  having  reached  a  solution.  When  compared  to  the  other  algorithms, 
discussed  in  chapter  3,  this  approach  is  very  pragmatic,  but  very  effective.  In  this  case,  conflia 
has  a  slightly  different  definition.  Shapiro's  use  of  the  term  correctness  lends  a  mote  formal 
principle  to  this  prot^m.  The  result  of  the  call  for  a  procedure  p  in  an  interpretation  is  correct,  if 
all  subgoals  of  p,  either  ditea  or  indirect,  yield  a  result  that  is  conea  in  M.  Shapiro's  algoritlmi 
yields  in  this  case  a  tripla  with  the  form  <p,  x,  y>.  that  is  the  cause  of  the  inconea  result  If  a 
misang  solution  is  the  subject  then  a  di^inctxm  must  be  matte  between  deterministic  (e.g.  Algol- 
derivatives  and  ftmctional  languages  such  as  Lisp)  and  rKxi-deterministic  languages  (e.g.  logic 
programming  Imguages  sudr  as  Prolog).  Suppose  that  y  is  die  conea  ou^  of  the  call  <p,  x>.  If 
<p.  x>  terminates  and  yields  a  diflerem  value  than  y  as  weD,  then  this  output  is,  in  case  of  a 
deterministic  language,  not  correct.  In  a  non-determinisdc  language  it  is  so  that  if  every  call  <p, 
x>  yields  an  output  that  is  correa  in  M,  but  na  a  single  call  yields  y,  dien  the  call  <p,  x,  y>  faQs. 
In  this  case,  the  procedure  pUt  not  confute.  A  program  P  is  complete  ui  il#  for  every  tripla  <p. 
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x,y>vnM,  if  there  is  a  call  <p,  x>  that  has  as  for  its  result.  If  the  piognun  P  fails  on  any  triplet 
<p,  X,  y>  in  M,  then  P  is  not  com|dete  in  M.  The  problem  can  be  solved  by  adapting  procedure  p, 
such  that  y  will  indeed  be  produced. 

It  is  not  very  well  possible  to  label  one  of  the  types  of  integrity  checks  as  an  all-purpose  winner. 
What  is  the  best  method  will  probably  dqrend  on  the  available  rtmning  time  and  memory  space. 
Also  the  question  how  large  a  knowledgebase  may  become  before  the  calculatioi  time  needed  for 
an  integrity  check  becomes  unacceptable  remains  unanswered,  because  the  various  algorithms 
could  not  be  tested  on  realistic  applications.  However,  an  answer  -reasonably  well-founded-  can 
be  given  to  the  question  which  algorithms  must  first  be  taken  into  account  for  further 
investigation  and  which  combination  is  expected  to  give  a  quality  result. 

The  deductive  database  approach  is  especially  suited  for  seomd  generatitm  expert  systems.  The 
internal  models  of  such  a  system  in  that  case  forni  a  fixed  (not  changing  per  session)  bank  of  facts. 
Even  when  one  puts  aside  a  production-rule  formalism,  a  deductive  database  offers  good 
possibilities  (thanks  to  Prolog’s  great  flexibility  and  the  fact  that  the  integrity  check  uses,  in  the 
last  place,  the  Prolog  interpreter).  Of  the  two  methods  discussed  in  this  category.  Decker's  metttod 
is  by  far  the  easiest  to  implement.  This  method  could  be  enhanced  with  a  Shapiro-like  circularity 
detector  and  possibly  the  completeness  check  like  that  of  Nguyen.  Furthermore,  it  is  absolutely 
necessary  that,  in  the  case  of  'user  askable'  parameters,  the  integrity  check  contains  all  possible 
(permitted)  combinations  of  parameter  values. 

Shapiro's  algorithm  offers  good  possibilities  for  the  integration  of  conceptual  modelling  (ENIAM) 
and  integrity  checking.  In  determining  non-termination,  it  is  felt  that  the  algorithm  lacks  more 
support  in  further  diagnosing  the  proof  tree  that  is  produced.  Adding  other  techniques  as  is 
suggested  by  Nguyen  is  worth  considering.  From  the  analysis  of  correctness  and  completoiess  it 
has  beonne  evident  that  the  user  plays  an  important  (interactive)  role.  By  selecting  variants  widi 
the  most  favouraUe  query  complexity  and  extending  the  algorithms  to  'full-fledged'  Prolog,  an 
integrated  AI  development  environment  can  be  realized.  The  posrilMlities  offered  by  the  Model 
InferetKc  System,  irreluding  automatic  repair  of  irKorrect  procedures,  can  certaitdy  play  a  role  in 
this. 
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7  CONCLUSIONS 

The  quality  framework  as  presented  at  the  beginning  of  this  study,  forms  an  adequate  guideline 
for  the  iq>pioach  of  quality  problems  in  expert  systems.  In  concrete  terms,  diis  means  diat  the 
notion  of  quality  will  be  looked  at  from  three  angles:  the  system  qtecificatitms,  die  structuring  of 
the  developmertt  process  and  the  expert  system  as  a  product  Further  elaboration  of  validation  and 
verification  methods  and  techniques  for  specifications  as  well  as  the  final  product  have  been 
reviewed.  Furthermore,  guidelines  have  been  given  for  a  structured  development  process  of 
expert  systems  that  can  be  properly  controlled. 

As  the  most  important  part  of  an  expert  system,  the  knowledgebase  has  been  mentioned. 
Reasoning  mechanism,  man-machine  interface  and  the  explanation  facility  contain  so  many 
conventional  components  that  quality  assurance  can  take  place  with  many  methods  and 
techniques  that  are  already  widely  used. 

Reliability  and  maintainability  are  the  most  important  criteria  associated  with  expert  systems 
development. 

The  solution  of  the  aforementioned  problems  encountered  in  practice,  must  in  the  first  place  be 
searched  in  improving  the  modularity  and  integrity  of  knowledgebases. 

Two  technologies  from  the  realm  of  software  engineering  seem  to  occupy  the  best  slot  to  help 
realize  these  improvements.  Especially  database  technology  is  able  to  contribute  the  necessary 
concepts  and  methods  for  controlling  integrity.  It  is  true  that  not  all  integrity  constraints  necessary 
for  knowledgebases  can  be  traced  back  to  ctmventional  database  systems,  but  the  methods  and 
techniques  designed  for  checking  integrity  in  deductive  database  systems  are  so  wide  that 
integrity  constraints  that  fall  under  the  denominator  of  consistency  and  comfdeteness  are  covered. 
Furthermore,  die  use  of  database  tedmology  offers  additional  advantages  as  far  as  security, 
recovery,  concurrency  and  distribution  are  concerned. 

For  the  apfdication  of  a  modular  structure,  database  technology  is  smneudiat  less  suitabk. 
Bladtboard  technology  seems  in  this  respect  to  be  a  better  alternative. 
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The  development  process  of  expeit  systems  is  difficult  to  control  Contrary  to  {biasing  methods 
for  conventitMud  system  development,  there  is  no  such  ai^roach  for  expert  systems  that  is 
generally  accepted.  The  experience  that  is  gained  over  the  last  few  years  in  using  mediods  like 
SKE  and  a  phasing  as  in  the  Weitzel-Kerschberg  life  cycle  model,  will  in  the  future  possibly  offer 
some  alleviation.  Quality  assurance  is  only  possible  if  a  feasibility  study  is  made  in  the  first 
phase,  just  as  it  is  done  in  conventional  system  development 

As  yet,  there  is  no  suitable  (and  tested)  method  that  can  be  used  in  making  clear  Reifications  of 
expert  systems.  The  development  of  KADS  qualifies  as  an  incentive,  but  the  types  of  expert 
systems  that  lend  themselves  for  the  application  of  this  method,  remain  limited.  It  is  possible  that 
this  problem  can  be  solved  to  a  certain  extent  by  using  an  extended  conventional  method,  such  as 
ENIAM.  This  will  also  involve  the  necessary  attention  to  the  way  in  which  knowledge  will  be 
elicited  from  the  expert 

E(xtcndcd)NlAM  offers  better  possibilities  to  record  the  specifications  of  a  knowledgebase  than 
NIAM  does.  The  possibilities  to  actually  represent  non-gra{duc  constrainu  of  NIAM  in  a  gitqjhic 
way,  in  particular,  is  of  great  importance  to  the  expert  for  directly  validating  the  conceptual 
model.  It  is  unfortunate,  however,  that  the  other  shortcoming  of  NIAM  has  not  been  dealt  with  yet 
in  the  E(xtended)  version,  namely  the  explicit  registration  of  temporal  aspects. 

Evaluating  and  testing  expert  systems  is  as  yet  an  underdeveloped  field.  Often,  it  is  impossible  to 
test  whether  the  advice  given  by  an  expert  system  complies  with  an  objective  standard.  For  the 
purpose  of  user  acceptance  it  is  of  great  importance  to  ktx>w  in  how  far  a  system  meets  the 
requirements  and  thus  forms  a  quality  aspect  Test  cases  will  have  to  be  designed  but  a  structured 
approach  to  do  this  has  as  yet  not  been  developed  for  expert  systems.  Application  of  traditional 
testing  methoddogies  may  offer  a  solution  in  this  respect 

The  deductive  datrtrase  appiuach  pays  too  little  attention  (actually  tKxie  at  all)  to  drcularity  and 
incompleteness  when  compared  widi  die  Rfoaches  of  Nguyen  and  Shapiro.  This  is  balmced  by 
the  fact  diat  semantic  (deductive)  database  integrity  comprises  much  more  than  Nguyen's 
contradictions.  Furthermore,  si  algoridm  like  ths  of  Decker  can  be  easQy  implememed  in 
Prolog.  By  simpiifyii^  constraints  itw>  iqxiate  constrairtts  and  especially  using  die  meta* 
inftmnation  about  derivation  rules,  computational  complexity  will  be  substandally  reduced.  The 
price  that  must  be  paid  is  die  multi{dicaiion  ot  die  necessary  memory  spuce:  an  avoage  of  k 
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literals  per  rule-body  needs  about  k-i-l  times  of  memory  space  to  store  the  meta-infoimtuion 
(occurs Jn  predicates)  together  with  the  original  database  and  its  copy. 

The  aim  of  Shapiro's  algorithm  was  to  design  a  formal  theory  for  the  debug^ng  process.  The  use 
of  Prolog  was  instigated  by  the  citcumstaiKe  that  titere  is  a  direct  relationship  betweoi  the  syntax 
(structure)  and  the  semantics  (meaning)  of  Prolog.  This  makes  it  simple  to  diagnose  and  correct 
faulty  programs.  This  characteristic  is  very  welcome  in  the  analysis  of  knowledgebase 
specifications,  because  a  Prolog  program  may  be  seen  as  a  formalization  of  these  specifications. 
So  it  is  more  than  just  a  program.  However,  it  must  be  noted  that  this  study  has  only  dealt  with 
'pure'  Prolog.  An  extension  in  the  direction  of 'full-fledged'  Prolog  is  very  well  feasible,  according 
to  Shapiro  [Shapro84]. 

By  using  Shapiro's  algorithm  it  is  possible  to  investigate  a  knowledgebase  specification  that  can 
be  executed  in  Prolog,  as  far  as  consistency  and  completeness  are  concerned.  However,  it  will  be 
iwcessary  that  the  conceptual  model  made  with  the  help  of  ENIAM  can  be  transposed  into  a 
Prolog  program. 

The  combination  of  ENIAM,  Prolog  and  the  algorithms  of  Shapiro  and  Decker  enables  the 
developer  of  knowledgebase  specifications  to  work  on  a  consistent  and  comi^ete  conceptual 
model  of  a  problem  domain  within  the  same  theoretical  framework  (incrementally).  The  fact  that 
the  method  (ENIAM)  acts  as  a  starting  point,  makes  it  a  sturdier  approach  than  that  of  Nguyen.  It 
is  true  that  the  latter  performs  a  very  extensive  analysis  of  a  knowledgebase,  but  (as  yet)  is  not 
interested  in  the  underlying  (conceptual)  model  of  reality. 

Although  these  technologies  will  in  most  cases  show  a  significant  improvement  in  quality,  they 
are  not  capable  of  solving  all  quttiity  problems  in  expert  system  software.  Knowledge  modelling 
in  particular,  will  always  have  be  a  problem  for  certain  domains  urd  as  such  they  ate  labelled 
pathologic.  This  lefns  to  so-called  pre-paradigmatic  knowledge  domains;  stmidiow,  th^ 
domains  have  not  (yet)  been  provided  with  exfrficit  theory  in  w  orderly  way.  As  a  practical 
exanpk  a  medicid  domain  comes  to  mirrd,  with  concqits  that  often  ate  not  sharply  defined  and  a 
myriad  of  synonyms  and  near-synonyms  further  obscuring  a  dear  view.  In  such  a  domain,  the 
knowledge  engfnetx  faces  the  task  of  building  a  coheieiK  «id  compact  theory.  Needless  to  say 
that  for  aach  n  mentiaUy  creative  tadt  no  ready-made  answms  may  be  mqieciBd  -nm  ftom  any 
tedmotogy-,  other  than  a  certain  amotmt  of  support  Ndtfier  dmdMse  nor  biackboaid  techndogy 
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can  offer  a  lot  of  support  in  this  respect  In  some  cases  the  translatitm  of  concepts  from  artother 
domain  can  possibly  offer  a  solution. 
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